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1. Purpose 

On Thursday, September 23, 2010, the Research and Science Education Sub- 
committee will hold a hearing to examine the current state of science and tech- 
nology policy research, how this research informs policymaking, and the role of the 
federal government in fostering academic research and education in this emerging 
interdisciplinary field. 

2. Witnesses 

• Dr. Julia Lane, Program Director of the Science of Science and Innovation 
Policy program. National Science Foundation. 

• Dr. Daniel Sarewitz, Co-Director of the Consortium for Science, Policy & 
Outcomes, Arizona State University. 

• Dr. Fiona Murray, Associate Professor of Management in the Technological 
Innovation & Entrepreneurship Group, MIT Sloan School of Management. 

• Dr. Albert H. Teich, Director of Science & Policy Programs, American Asso- 
ciation for the Advancement of Science. 

3. Overarching Questions 

• What is the “science of science policy?” How can science and technology (S&T) 
policy research contribute to and inform evidence-based local and national 
policy decisions? To what extent are science and technology policies in the 
United States being shaped by what has been learned from S&T policy re- 
search? 

• What new and continuing areas of research in this area could significantly 
improve our ability to design effective programs and better target federal re- 
search investments? What are the most promising research opportunities and 
what are the biggest research gaps? Is the Federal government, specifically 
the National Science Foundation, playing an effective role in developing the 
science of science policy? 

• What is the state of education in science and technology policy at U.S. univer- 
sities? What are the backgrounds of students pursuing graduate degrees in 
S&T policy? What career paths are sought by science and technology policy 
program graduates? What are the fundamental skills and content knowledge 
needed by science and technology policy practitioners? Is the National Science 
Foundation playing an effective role in fostering the development of science 
and technology policy programs at U.S. universities? 

4. Background 

During his keynote address in 2005 at the American Association for the Advance- 
ment of Science’s Science and Technology Policy Forum, Dr. John Marburger, then 
science advisor to President Bush, called for the establishment of a “science of 
science policy.” The “science of science policy” (SoSP) as described by Dr. Marburger 
and others includes the development of scientific theories, analytical tools, and rig- 
orous datasets that will assist policymakers in science policy decisions. The SoSP 
is an interdisciplinary field that draws together researchers from economics, polit- 
ical science, and the social and behavioral sciences to improve our understanding 
of the science and engineering enterprise, including the process of innovation in an 
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effort to establish a more quantitative approach to science and technology policy de- 
cisions. 

While most believe that science, technology, and innovation are critical to the 
competitiveness and prosperity of the United States, we lack the rigorous tools to 
quantify that relationship. Therefore, it remains difficult to actually measure the 
economic impact, social benefits, and effectiveness of federal research and develop- 
ment (R&D) investments. In addition to improving our ability to target federal R&D 
investments, research in the area of SoSP holds the potential to provide insight into 
the effect of globalization on the U.S. science and engineering workforce, increase 
our understanding of technology development and diffusion, communicate the social 
and economic benehts of R&D spending to the general public, and shed light on the 
process of creativity and innovation. 

In 2006, in response to Dr. Marburger’s call to action, an interagency working 
group, co-chaired by the National Science Foundation (NSF) and the Department of 
Energy, was formed within the Subcommittee on Social, Behavioral, and Economic 
Sciences under the National Science and Technology Council. The interagency work- 
ing group conducted an assessment of the state of SoSP research and surveyed the 
Federal agencies about the tools, methods, and data they were using to make invest- 
ment decisions. This work resulted in the release of a Federal SoSP research road- 
map 1 in 2008. The roadmap outlines three broad themes and poses 10 research 
questions to be addressed by federally-funded SoSP research. 

Theme 1: Understanding Science and Innovation 

Question 1: What are the behavioral foundations of innovation? 

Question 2: What explains technology development, adoption, and diffusion? 

Question 3: How and why do communities of science and innovation form and 
evolve? 

Theme 2: Investing in Science and Innovation 

Question 4: What is the value of the Nation’s public investment in science? 

Question 5: Is it possible to “predict discovery”? 

Question 6: Is it possible to describe the impact of discovery on innovation? 

Question 7 : What are the determinants of investment effectiveness? 

Theme 3: Using the Science of Science Policy to Address National Priorities 

Question 8: What impact does science have on innovation and competitiveness? 

Question 9: How competitive is the U.S. scientihc workforce? 

Question 10: What is the relative importance of different policy instruments in 
science policy? 

Role of the National Seience Foundation 

In 2006, NSF’s Directorate for Social, Behavioral and Economic Sciences (SBE) 
held three workshops to ask for recommendations and guidance from the research 
community about the breadth of activities that should be supported under an NSF- 
funded SoSP program. In 2007, NSF allocated $6.8 million for a new Science of 
Science and Innovation Policy (SciSIP) program. SciSIP supports both single inves- 
tigators and collaborations in two areas. First, the program supports research on 
data and the improvement of science metrics, including research to improve our 
ability to identify, characterize, and measure returns on federal R&D investments. 
Second, the program supports research directed toward the development of models 
and other statistical tools as well as qualitative studies that will improve our under- 
standing of the process of innovation and science outcomes, both societal and eco- 
nomic. In addition to supporting research, the program supports workshops, con- 
ferences, and symposia to help foster a community of researchers in the SciSIP area. 

NSF’s SciSIP budget request for fiscal year 2011 was $14.25 million, of which 
$8.05 million will be devoted to SciSIP research and community building activities 
through SBE’s Office of Multidisciplinary Activities and $6.2 million will be for the 
development of data survey tools through SBE’s Division of Science Resource Statis- 
tics (SRS). The data compiled by SRS for the biennial Science and Engineering Indi- 


^The Science of Science Policy: A Federal Research Roadmap http:! j www.whitehouse.gov j 
files i documents / ostp I NSTC%20Reports j 39924-PDF%20Proof.pdf 
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cators report serve a vital role in the SoSP as a long-term source of unbiased infor- 
mation about the science and engineering enterprise. 

NSF’s current efforts in SciSIP are not its first. From the 1970’s through the early 
1990’s NSF had a modest-sized staff carrying out policy research and analysis. 
These analysts worked in the Office of Research and Development Assessment, later 
the Division of Policy Research and Analysis (PRA), on specific tasks requested by 
the Office of Management and Budget, the Office of Science and Technology Policy, 
the Congressional Office of Technology Assessment, and other federal agencies. Ad- 
ditionally, PRA had a small budget to support academic research in areas directly 
relevant to their policy analysis tasks. In 1992, PRA was involved in a scandal over 
the faulty assumptions used to predict a looming shortage in engineers. The scandal 
led to an investigation by the Committee on Science & Technology and the disman- 
tling of PRA. 

STAR METRICS 

The National Science Foundation and the National Institutes of Health are cur- 
rently collaborating on a project known as STAR METRICS (Science and Technology 
for America’s Reinvestment: Measuring the Effect of Research on Innovation, Com- 
petitiveness and Science), which is the first federal-university partnership to de- 
velop a data infrastructure that documents the outcomes of science investments for 
the public. An initial pilot project was recently completed with a handful of region- 
ally and otherwise diverse institutions of higher education through the National 
Academies’ Federal Demonstration Partnership. The pilot project validated the ini- 
tiative’s concept and its ability to collect relevant data from existing university data- 
bases. The full-scale project will proceed in two phases: Phase I will develop uni- 
form, auditable and standardized measures of job creation resulting from science 
spending included in the American Recovery and Reinvestment Act; Phase II will 
develop measures of the impact of federal R&D spending on economic growth, work- 
force development, scientific knowledge, and social outcomes. 

International Efforts 

The Organization for Economic Cooperation and Development (OECD) has been 
developing and collecting science and technology indicators from their member na- 
tions for nearly 50 years. In 2004, the Science & Technology Ministerial called for 
a “new generation of indicators which can measure innovative performance and 
other related output of a knowledge-based economy” emphasizing “the data required 
for the assessment, monitoring and policy making purposes.”^ Since that time the 
OECD has continued to refine its science and technology indicators, and improve the 
tools they use for analyzing the impact of science and technology. Earlier this year 
the OECD released a report entitled, “Measuring Innovation: A New Perspective.”^ 
The report identifies five areas for which international action is needed: the develop- 
ment of innovation metrics that can be linked to aggregate measures of economic 
performance; investment in a high-quality and comprehensive statistical infrastruc- 
ture to analysis innovation at the firm-level; the promotion of innovation metrics in 
public sector and for public policy evaluation; the identification of new approaches 
to understand knowledge creation and flow; and the promotion of the measurement 
of innovation on social goals. 

On April 14, Dr. Julia Lane spoke to the European Parliament about the STAR 
METRICS effort, emphasizing the global nature of science and engineering and the 
common need for better tools to assess and predict the impact of science, technology, 
and innovation. During her speech, Dr. Lane indicated that creating a universal re- 
searcher identification system could be an important first step in a global effort to 
understand and measure the return on scientific investment. Niki Tzavela, a Greek 
Member of the European Parliament, who serves as Vice-Chair of the European 
Parliament Delegation to the United States, and sits on the Parliament’s Industry, 
Research, and Energy Committee (ITRE), has been a leader on the issue of im- 
proved science metrics in the European Union. Having indicated that the EU 8th 
Framework Program represents an opportunity to evaluate and improve science pol- 
icy, Mrs. Tzavela introduced an initiative to the ITRE Committee proposing that the 
EU collaborate on this topic with the United States. The EU is now considering ini- 
tiatives that would complement the STAR metrics project, and the Scientific Tech- 


2 What Indicators for Science, Technology and Innovation Policies in the 21st Century? Blue 
Sky II Forum — Background http: ! j wivw.oecd.org I dataoecd j 9 1 48 j 37082579.pdf 
^ http: 1 1 www.oecd.org j document 1 22 j 0,3343, en ^41462537 ^41454856-44979734^1 ^ 

l,00.html 
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nology Options Assessment Panel within the EU has been designated to provide an 
in-depth analysis on Science Metrics.'^ 

Education in Science & Technology Public Policy 

According to the AAAS Guide to Graduate Education in Science, Engineering and 
Public Policy® there are more than 25 U.S. universities that offer a graduate degree 
in the interdisciplinary field of science and technology public policy. These degree 
programs draw from a number of fields, including economics, sociology, political 
science, and engineering; however the coursework associated with each program 
varies and is dependent upon the academic department or school that houses the 
program. 

5. Questions for Witnesses 

Dr. Julia Lane 

1. Please describe NSF’s Science of Science Policy and Innovation program, in- 
cluding a description of the Foundation’s overall vision and strategy for re- 
search and education in this area. 

Specifically, 

How is NSF fostering collaboration between social and behavioral sci- 
entists and researchers from other disciplines, including computer sci- 
entists, engineers, and physical scientists, in science and technology 
policy research? 

How is NSF fostering the development of science and technology policy 
degree programs and courses of study at colleges and universities? 
what is the current scope and level of support for such programs? 

How is NSF encouraging the development of a community of practice 
in science of science policy and the dissemination of research results to 
policy makers? 

2. As a Co-Chair of the Science of Science Policy Interagency Group under the 
National Science and Technology Council, please briefly describe the work of 
that group and how the various federal science agencies are collaborating on 
the development and implementation of science of science policy tools to im- 
prove the management and effectiveness of their R&D portfolios and other 
science and technology-related programs. 

3. Please provide a brief description and update on the status of the OSTP led 
project on science metrics, known as STAR Metrics, including a description 
of international engagement and interest in this effort. 

Dr. Albert Teich 

1. How can research on innovation and the scientific enterprise also known as 
the science of science and innovation policy (SciSIP) be used to inform the 
design of effective federal programs and the management of federal research 
investments? Do you believe the results of science and technology policy re- 
search are being effectively incorporated into national policy decisions? 

2. What are the challenges to the incorporation of science and technology re- 
search in the decision making process? What is AAAS’s role in mitigating 
those barriers? Specifically, how is AAAS helping to build a community of 
practice in the SciSIP? What recommendations, if any, do you have for the 
National Science Foundation’s SciSIP program? Do you believe SciSIP re- 
search is being effectively coordinated across the federal agencies? If not, 
what if any recommendations do you have regarding interagency coordina- 
tion? 

3. As you know there are more than 25 U.S. universities that offer graduate 
degrees in science, engineering and public policy. In your opinion, are these 
programs having the intended effect of producing graduates with the skills 
necessary to shape science and technology policies? What type of education 
and training should science and technology policy practitioners receive? Is 
the National Science Foundation playing an effective role in fostering the de- 
velopment of science and technology policy programs at U.S. universities? If 


I www.euractiv.com I en I science I eu-looks-to-us-model-for-measuring-rd-impact-news- 

448950 

^http:! I www.aaas.org / spp / sepp I 
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not, what recommendations, if any, do you have for NSF and/or the univer- 
sities with such programs? 

Dr. Daniel Sarewitz 

1. Please provide an overview of the research activities of the Consortium for 
Science, Policy, and Outcomes. How are you facilitating interdisciplinary col- 
laborations within the Consortium? What new and continuing areas of re- 
search in the science of science and innovation policy (SciSIP) could signifi- 
cantly improve our ability to design effective programs and better target fed- 
eral research investments? What are the most promising research opportuni- 
ties and what are the biggest research gaps? 

2. Is the Federal government, specifically the National Science Foundation, 
playing an effective role in fostering SciSIP research and the development 
of a community of practice in SciSIP? What recommendations, if any, do you 
have for the National Science Foundation’s SciSIP program? 

3. Please describe the education and outreach activities of the Consortium for 
Science, Policy, and Outcomes. 

4. How can the dissemination of SciSIP research findings be improved so that 
policymakers are better informed of the current state of research? Are there 
best practices that can be implemented by the Federal government and/or 
the research community to improve the incorporation of science and tech- 
nology policy research into the decision making process? 

6. What are the fundamental skills and content knowledge needed by SciSIP 
researchers and practitioners? What are the backgrounds of students pur- 
suing graduate degrees in science and technology policy and what careers 
paths are sought by these graduates? Is the National Science Foundation 
playing an effective role in fostering the development of science and tech- 
nology policy degree programs at U.S. universities? If not, what rec- 
ommendations, if any, do you have for NSF and/or the universities with such 
programs? 

Dr. Fiona Murray 

1. Please provide an overview of your research. What new and continuing areas 
of research in the science of science and innovation policy (SciSIP) could sig- 
nificantly improve our ability to design effective programs and better target 
federal research investments? What are the most promising research oppor- 
tunities and what are the biggest research gaps? 

2. Is the Federal government, specifically the National Science Foundation, 
playing an effective role in fostering SciSIP research and the development 
of a community of practice in SciSIP? What recommendations, if any, do you 
have for the National Science Foundation’s SciSIP program? 

3. What are the fundamental skills and content knowledge needed by SciSIP 
researchers and practitioners? What are the backgrounds of students pur- 
suing graduate degrees in science and technology policy and what careers 
paths are sought by these graduates? Is the National Science Foundation 
playing an effective role in fostering the development of science and tech- 
nology policy degree programs at U.S. universities? If not, what rec- 
ommendations, if any, do you have for NSF and/or the universities with such 
programs? 
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Chairman LiPiNSKi. This hearing will now come to order. Good 
afternoon and welcome to today’s Research and Science Education 
Subcommittee hearing on the Science of Science and Innovation 
Policy, also known as SciSIP. For those of you who may not be fa- 
miliar with the phrase, the Science of Science Policy is a field of 
interdisciplinary research that focuses on understanding how our 
policy decisions impact innovation and science and engineering re- 
search. Given the magnitude of the federal investment in science 
and technology, there is a need for objective analysis and evalua- 
tion of federally funded R&D programs. And given the size of the 
budget deficit. Congressional decision makers need the best infor- 
mation possible to make sure we are spending taxpayer dollars op- 
timally. 

Today we will be hearing from a diverse panel of witnesses about 
the current state of research and education in this emerging field. 
This topic is of particular interest to me since it goes to the core 
of why I joined the Science and Technology Committee when I first 
came to Congress. Like most members of this committee, I believe 
that science and engineering research, and education have driven 
long term economic growth and improved the quality of life for all 
Americans. I have viewed science and innovation policy as critical 
for maintaining our international competitiveness and creating 
jobs. 

But the best policies are not self-evident. As someone who was 
trained as an engineer and a social scientist, I believe we need data 
and proper analysis of this data to be able to determine as best we 
can the optimal policy. We are going to hear today about some of 
the research that is being done on science policy. I am eager to 
hear to the panel’s thoughts on what is being found, how well these 
findings are being disseminated, and whether research in this area 
is actually helping policymakers. 

While many of us would agree that science has had a positive im- 
pact on our lives, I think we know very little about how the process 
of innovation works. What kinds of research programs or institu- 
tional structures are most effective? How do investments in R&D 
translate to more jobs, improved health, and overall societal well- 
being? How should we balance investments in basic and applied re- 
search? With millions of Americans out of work it becomes more 
critical than ever that we find answers to these questions. 

We will also take a closer look at the state of education in science 
and technology policy and how these degree programs and courses 
of study are contributing by educating the next generation of re- 
searchers and science policy practitioners. There are a variety of 
science and technology programs that are popping up across the 
country. They can be found in public policy schools, economics de- 
partments, business schools, and other places, even philosophy de- 
partments. I am looking forward to hearing more about these pro- 
grams, including what kinds of students they attract and where 
those students go upon graduation. 

Finally, I hope to hear recommendations from today’s witnesses 
about how the Federal Government, particularly the National 
Science Foundation, can foster interdisciplinary research in this 
area, and how it can continue to improved education and training 
for students who want to pursue a career at the intersection of 
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science, technology, and public policy. I thank the witnesses for 
being here this afternoon, especially as we have had to move this 
hearing back from the morning. I look forward to your testimony. 

Now before I recognize Dr. Ehlers, I — this will likely be the last 
hearing of this subcommittee, the last meeting of this sub- 
committee. It may not be, but just in case it is the last for this Con- 
gress, I wanted to say that I think we should all recognize Dr. 
Ehlers for his contributions in Congress and especially on this com- 
mittee through the years. It has been certainly — I have had a great 
partner working on this as I have chaired the Subcommittee for the 
last two years. He is someone who really, truly is dedicated to the 
issues that we are facing here and we deal with here in the Com- 
mittee. Too many things right now are becoming partisan footballs, 
and Dr. Ehlers really has kept his eye on what is best and trying 
to find what is best for our country. And I want to thank you for 
the years that you have put in here and wish you the best in your 
next endeavors, but it has been a pleasure to work with you, espe- 
cially over these last few years. 

[The prepared statement of Chairman Lipinski follows:] 

Prepared Statement of Chairman Daniel Lipinski 

Good afternoon and welcome to today’s Research and Science Education Sub- 
committee hearing on the science of science and innovation policy, also know as 
SciSIP. For those of you who might not be familiar with the phrase, the “science 
of science policy” is a field of interdisciplinary research that focuses on under- 
standing how our policy decisions impact innovation and science and engineering re- 
search. Given the magnitude of the federal investment in science and technology, 
there is a need for objective analysis and evaluation of federally funded R&D pro- 
grams. And given the size of the budget deficit, Congressional decision makers need 
the best information possible to make sure we are spending taxpayer dollars opti- 
mally. Today we’ll be hearing from a diverse panel of witnesses about the current 
state of research and education in this emerging field. 

This topic is of particular interest to me since it goes to the core of why I joined 
the Science and Technology Committee when I came to Congress. Like most Mem- 
bers of this committee, I believe that science and engineering research and edu- 
cation have driven long-term economic growth and improved the quality of life for 
all Americans. I view science and innovation policy as critical for maintaining our 
international competitiveness and creating jobs. 

But the best policies are not self-evident. As someone who was trained as an engi- 
neer and a social scientist, I believe we need data and proper analysis of this data 
to be able to determine — as best we can — the optimal policy to implement. We are 
going to hear today about some of the research that is being done on science policy, 
and I am eager to hear the panel’s thoughts on what is being found, how well the 
findings of this research are being disseminated, and whether research in this area 
is actually helping policy makers. 

While many of us would agree that science has had a positive impact on our lives, 
I think we actually know very little about how the process of innovation works. 
What kinds of research programs or institutional structures are most effective? How 
do investments in R&D translate to more jobs, improved health, and overall societal 
wellbeing? How should we balance investments in basic and applied research? With 
millions of Americans out of work, it becomes more critical than ever that we find 
answers to these questions. 

We’ll also take a closer look at the state of education in science and technology 
policy and how these degree programs and courses of study are contributing by edu- 
cating the next generation of researchers and science policy practitioners. There are 
a variety of science and technology policy programs that are popping up across the 
country. They can be found in public policy schools, economics departments, busi- 
ness schools, and other places, even philosophy departments. I’m looking forward to 
hearing more about these programs, including what kind of students they attract 
and where those students go upon graduation. 

Finally, I hope to hear recommendations from today’s witnesses about how the 
Federal government, particularly the National Science Foundation, can foster inter- 
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disciplinary research in this area and how it can contribute to improved education 
and training for students who want to pursue a career at the intersection of science, 
technology, and public policy. 

I thank the witnesses for being here this afternoon and look forward to their testi- 
mony. 

Chairman LiPiNSKl. And with that I will now recognize Dr. 
Ehlers for an opening statement. 

Mr. Ehlers. Thank you for those kind words, Mr. Chairman. I 
think my biggest challenge will be learning how to sleep in, but I 
very much appreciate those comments. I always just try to do a 
good job wherever I am. It is a trade I learned from my parents 
and I never, never, never, ever expected to be in the Congress or 
in politics. My mother never quite got over it. As she put it, what 
are you doing with all those nasty people? But it turns out my col- 
leagues are not nasty people, and I appreciate your leadership on 
the Subcommittee. And you have done a great job of leading us in 
the right direction, and it has been a pleasure to work with you. 
Thank you. 

With that I will proceed to the opening statement. Today we will 
explore the current state of science and technology policy research 
and the role it plays in informing our policy decisions. And I have 
to insert a little comment in here, that is, when I first arrived here 
and was assigned to the Science Committee, which made obvious 
sense since I was at that time I think one of the very few, if the 
only, scientist in the Congress, at least on the Republican side. And 
at the first meeting of the Science Committee I asked the Chair, 
how many scientists do you have on staff? And the answer was 
none. And I said really? How can you function without — and he 
said well, we don’t really need people who understand science. We 
need people who understand science policy. 

Well, as a scientist I had never thought much about science pol- 
icy and little did I know that in conversation with Newt Gingrich 
where I commented that I thoug:ht it a bit strange that the science 
policy we were operating under in the government and in the Con- 
gress was by Vannevar Bush’s 1945 book, and I said that is a little 
out of date. Things change rapidly in science. It is a great book, 
“The Endless Frontier”. Vannevar Bush was a great man. He had 
done a lot of good works especially during World War II. But I 
talked to Newt Gingrich about that, that that was the latest 
science policy book that was guiding the Government, so he did as 
Newt Gingrich always did and said hey, it is time to get another 
one. Why don’t you do it? So I learned after a couple of years never 
to suggest anything to Newt because he always dumped the burden 
on me. 

In any event, I did proceed to work on a book, which just walked 
in the door with my aide, and some of you have seen it already. 
It is called “Unlocking Our Future”. Now this is not a great science 
policy book. I knew absolutely nothing about science policy when 
we proceeded to write it but it seemed to me that there were cer- 
tain things that were obvious and we put them in here. And I de- 
liberately said “Unlocking Our Future” because I felt we had so 
much to do and I was not able to do it in this thin little volume. 
It did get some notice, and it inspired some science policy individ- 
uals to engage more seriously in this. And some of them, many of 
them are represented here. But it was a real education to me. You 
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should try that sometime, writing a book about something you 
know nothing about. It is a great way to learn and fortunately as 
a child I was homeschooled because of illness, so all the learning 
I did was things that I learned and learned on my own. So that 
was good preparation for this. 

We clearly, badly need something like this again and it is one 
case where the author is delighted to say this is too old now. It is 
time to get busy. Someone else better start writing a better thing. 

Let me continue with my opening statement. When Dr. John 
Marburger, who was science advisor to President Bush, called for 
the establishment of a Science of Science Policy in 2005, we em- 
barked on a new journey into this emerging field of interdiscipli- 
nary research by establishing an interagency working group, the 
Science and Science of Innovation Policy program at the National 
Science Foundation — the shorthand for it was SciSIP — and most re- 
cently, the Science and Technology for America’s Reinvestment, 
which is the emphasis that I and others, including the authors of 
“The Gathering Storm,” have been emphasizing, because it is im- 
portant for us to measure the effects of research and innovation 
competitiveness and science which has come to be called STAR 
METRICS. I hope this hearing will provide us with a detailed 
measurement of how far we have come on that journey as well as 
an encouraging picture of the progress we have made. 

I have spent many years on this Committee working to strength- 
en U.S. innovation and science education, and I have been a long 
time advocate of increased federal funding for basic research. I 
wish the entire Congress was receptive to that notion as they — but 
this funding produces the technological innovations that will keep 
America competitive in the global market and it is essential for us 
to educate American workers in the skills needed for 21st century 
jobs. 

As with any program, sustained Congressional oversight is re- 
quired to insure that the Science of Science Policy Programs are ef- 
fective and that they progress in a timely and fiscally responsible 
manner. I am encouraged by efforts which seek to maximize our 
current investments in scientific research, and I believe it is very 
important that those are the investments that provide us with 
measureable returns. And that is why I have worked so hard to try 
and make the research and development tax credit permanent, be- 
cause that is one good way to encourage industry to work on these 
issues. We must be mindful of that fact as Congress deliberates the 
best ways to use American taxpayer funds in this difficult economic 
climate. 

To that end I am very interested in learning more about the 
progress and potential of the STAR METRICS program and its re- 
cently completed project. I hope — I look forward to learning more 
about the status of science affecting science policy and the advance- 
ments which have been made since 2005. And I wanted to thank 
our panel of witnesses for being here today, for accommodating our 
last second scheduling change, and I look forward to hearing their 
insights on this topic. There is much work to be done to help our 
nation recover its lead in technological development, and in manu- 
facturing, and in science in general. 
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And so I am looking forward to the testimony today and I hope 
you can enlighten us, and out of this will come first of all a new 
version of this, and secondly there is some improvement in our 
judgments about science and also science education in this Nation. 
Thank you very much. 

[Statement of Mr. Ehlers follows:] 

Prepared Statement of Representative Vernon J. Ehlers 

Today, we will explore the current state of science and technology policy research, 
and the role it plays in informing our policy decisions. When Dr. John Marhurger, 
then science advisor to President Bush, called for the establishment of a “science 
of science policy” (SoSP) in 2005, we embarked on a new journey into this emerging 
field of interdisciplinary research by establishing an interagency working group, the 
Science of Science and Innovation Policy (SciSIP) program at the National Science 
Foundation (NSF), and, most recently, the Science and Technology for America’s Re- 
investment: Measuring the Effect of Research on Innovation, Competitiveness and 
Science (STAR METRICS) project. I hope this hearing will provide us with a de- 
tailed measurement of how far we have come on that journey, as well as an encour- 
aging picture of the progress we have made. 

I have spent many years on this committee working to strengthen U.S. innovation 
and science education, and I have been a long time advocate of increased federal 
funding for basic research. This funding produces the technological innovations that 
will keep America competitive in the global market, and it is essential for us to edu- 
cate American workers in the skills needed for 21st-century jobs. 

As with any program, sustained Congressional oversight is required to ensure the 
SoSP programs are effective, and that they progress in a timely and fiscally respon- 
sible manner. I am encouraged by efforts which seek to maximize our current in- 
vestments in scientific research, and I believe it is very important that those R&D 
investments provide us with measurable returns. We must be mindful of that fact 
as Congress deliberates the best ways to use American taxpayer funds in this dif- 
ficult economic climate. To that end, I am very interested in learning more about 
the progress and potential of the STAR METRICS program and its recently com- 
pleted pilot project. 

I look forward to learning more about the status of science affecting science policy 
and the advancements which have been made since 2005. I want to thank our panel 
of witnesses for being here today, for accommodating our last-second scheduling 
change, and I look forward to hearing their insights on this topic. 

Chairman LiPiNSKl. Thank you. Dr. Ehlers. Maybe we can make 
that a best seller now. Now if there are Members who wish to sub- 
mit additional opening statements, their statements will be added 
to the record at this point. And at this time I want to introduce 
our witnesses. Dr. Julia Lane is the Program Director of the 
Science of Science and Innovation Policy program at the National 
Science Foundation. Dr. Daniel Sarewitz is the Co-Director of the 
Consortium for Science, Policy & Outcomes and Professor of 
Science and Society at Arizona State University. Dr. Fiona Murray 
is an Associate Professor of Management in the Technological Inno- 
vation & Entrepreneur Group at MIT Sloan School of Management. 
And Dr. Albert H. Teich is the Director of Science & Policy Pro- 
grams at the American Association for the Advancement of Science. 

As our witnesses should know, you will each have five minutes 
for your spoken testimony. Your written testimony will be included 
in the record for the hearing. When you all have completed your 
spoken testimony we will begin with questions. Each Member will 
have five minutes to question the panel. And before we begin I just 
want to mention that we will be having votes coming up soon, so 
probably what the most important thing is if everyone could hold 
to their five minutes it will help us so we don’t have — hopefully 
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won’t have an interruption in the — at least in the testimony part 
here. But with that, we will start with Dr. Lane. 

STATEMENT OF JULIA LANE, PROGRAM DIRECTOR OF THE 

SCIENCE OF SCIENCE AND INNOVATION POLICY PROGRAM, 

NATIONAL SCIENCE FOUNDATION 

Dr. Lane. Chairman Lipinski, Ranking Member Ehlers, Members 
of the Subcommittee, it is my distinct pleasure to be with you here 
today to discuss NSFs Science of Science and Innovation Policy 
Program, the activities of the Science of Science Policy Inter-Agen- 
cy Group, and the STAR METRICS program, the last of which is 
a new federal and university partnership to document the sci- 
entific, social, economic, and work force outcomes of science invest- 
ments to the public. I am Dr. Julia Lane. I am the Program Direc- 
tor of the SciSIP program at NSE and Co-Chair of NSTC working 
group on the Science of Science Policy. 

I submitted a written statement to supplement or to accompany 
my oral testimony. So the focus of these three efforts is to provide, 
as you noted, better methods and data to inform science — federal 
science investment decisions. They represent the first efforts to 
construct a scientific framework that is supported by multiple 
agencies and multiple institutions, all jointly engaged. It represents 
a true all-out effort to providing an evidence basis for U.S. science 
policy. Its success is important for policy makers because, in a nut- 
shell, you can’t manage what you can’t measure, and what you 
measure is what you get. 

Developing a scientific framework involves several things. It re- 
quires the engagement of scientists from many disciplines to ad- 
dress science policy issues. The NSE does this through the SciSIP 
program. The overarching goal of this effort is to conduct basic re- 
search that creates new objective models, analytical tools, and data 
sets to inform our nation’s basic research, and to inform public and 
private sectors about the processes through which investments in 
science and engineering work their way through the outcomes we 
have mentioned. It funds researchers from a wide range of dis- 
ciplines and it funds students to study science policy issues in a 
scientific manner. As you also know, it supports the redesign of 
surveys undertaken by the National Science Eoundation’s Science 
Resource Statistics. It is the statistical agency charged with de- 
scribing the science and innovation enterprise. Eor example, the 
new business and innovation survey, the BRDIS [Business R&D 
and Innovation Survey] survey, has been completely redesigned 
from a 1950s structure to something that captures the new R&D 
innovation activities. 

So it is not just the academic community that is advancing the 
Science of Science Policy. It is also policymakers in the Executive 
and Legislative branches who recognize that we need these better 
approaches. That is why the National Science and Technology 
Council established the Science of Science Policy Inter-Agency Task 
Group. That task group, the science policy Agencies that were rep- 
resented on that, created a road map that characterized our cur- 
rent system of measuring the science and engineering enterprise as 
inadequate. We can do better. There is enormous potential to do 
better. The first step to doing better is to get better data. Just as 
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good bricks need straw, good research in an empirical field like 
science and innovation policy requires good data. So to that end, 
the SciSIP Program and the Science of Science Policy Inter-Agency 
Group initiated the development of the STAR METRICS program 
to which you have already alluded. The benefits of this program is 
that rather than having organized data sets that different agencies 
and different institutions use to answer the types of questions that 
the American people are asking, we can develop a common, bottom- 
up, empirical infrastructure that will be available to all recipients 
of federal funding and to science agencies to quickly respond to 
state, congressional, and 0MB requests. It is critical that we take 
a bottom-up effort in order to develop these approaches — one that 
is domain specific, generalized, and replicable. 

Phase one started in March, jointly sponsored by NIH, the lead 
agency, NSF, and OSTP. And that is collecting the data required 
to, with no burden, respond to questions about the jobs associated 
with science funding. Phase two, which is trying to collect broader 
data on a wide range of outcomes-not just jobs, but social, sci- 
entific, economic, and work force outcomes — is beginning this fall 
with formal consultations with research institutions. 

Furthermore, science is fundamentally an international endeav- 
or. We have engaged with the European Union, with the Japanese, 
with the Brazilians, with many countries in order to document the 
impact of science investments. In fact, the Japanese Government 
has recently set aside funding for a Japanese equivalent of a 
SciSIP program. The European Union has also shown considerable 
interest in what we have been up to, and is considering emulating 
the bottom-up, no burden endeavor that the SciSIP, the Science of 
Science Policy, and the STAR METRICS Program have pushed for- 
ward. And the Brazilian Government has also requested briefings 
on the SciSIP, Science of Science Policy, and STAR METRICS Pro- 
gram. 

This concludes my testimony, Mr. Chairman. I look forward to 
answering any questions you or the Members of the Committee 
might have. 

[The prepared statement of Dr. Lane follows:] 

Prepared Statement of Julia I. Lane 

Chairman Lipinski, Ranking Member Ehlers, and Members of the Subcommittee, 
it is my distinct privilege to be here with you today to discuss NSF’s Science of 
Science and Innovation Policy (SciSIP), the activities of the Science of Science Policy 
Interagency Group, and the STAR METRICS program — a new federal effort de- 
signed to create a scientific quantifiable measurement of the economic and social im- 
pacts of federal research spending. I am Dr. Julia Lane, the program Director of the 
SciSIP program at the National Science Foundation, and co-chair of the NSTC work- 
ing group on Science of Science Policy (SOCP). 

At the outset, I would like to express my appreciation to all the Members on the 
House Committee on Science and Technology for their unstinting support to advance 
both the cause, and the frontiers of science. This Committee has long held steadfast 
in the knowledge that America’s present and future strength, prosperity and global 
preeminence depend directly on fundamental research. 

The National Science Foundation has always believed that optimal use of limited 
Federal funds relies on two conditions: Ensuring that research is aimed — and con- 
tinuously re-aimed — at the frontiers of understanding; and certifying that every dol- 
lar goes to competitive, merit-reviewed, and time-limited awards with clear criteria 
for success. When these two conditions are met, the nation gets the most intellectual 
and economic leverage from its research investments. 
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Yet our portfolio keeps changing. We have a minimal vested interest in maintain- 
ing the status quo, and pride ourselves on an ability to shift resources quickly to 
the most exciting subjects and most ingenious researchers. 

Moreover, we regard it as an essential part of our mission to constantly re-think 
old categories and traditional perspectives. This ability is crucial now, because con- 
ventional boundaries are disappearing — boundaries between nations, boundaries be- 
tween disciplines, boundaries between science and engineering, and boundaries be- 
tween what is fundamental and what is application. At the border where research 
meets the unknown, the knowledge structures and techniques of life science, phys- 
ical science, and information science are merging. 

Additionally, our scope is extremely wide, extending across all the traditional 
mathematics, science and engineering disciplines. That is a major advantage in to- 
day’s research climate, where advances in one field frequently have immediate and 
important applications to another. The same mathematics used to describe the phys- 
ics of turbulent air masses may suddenly explain a phenomenon in ecology or in the 
stock market, or the changes in brain waves preceding an epileptic seizure. The 
same algorithms used by astronomers to discern patterns in the distant heavens can 
aid radiologists to understand a mammogram, or intelligence systems to identify a 
threat. Only an agency that sees the “big picture” can assure this kind of inter- 
disciplinary synergy. 

For all of these reasons, the National Science Foundation is fostering the develop- 
ment of the knowledge, theories, data, tools and human capital needed to cultivate 
a Science of Science and Innovation Policy program. The program has three major 
aims: advancing evidence-based science and innovation policy decision making; de- 
veloping and building a scientific community to study science and innovation policy; 
and developing new and improved datasets. 

The overarching goal in this effort, however, is to conduct basic research that cre- 
ates new objective models, analytic tools, and datasets to inform our nation’s public 
and private sectors about the processes through which investments in science and 
engineering research may be transformed into scientific, social and economic out- 
comes. 

We need to better understand the contexts, structures, and processes of scientific 
and engineering research, to evaluate reliably the tangible and intangible returns 
from investments in research and development (R&D), and to predict the likely re- 
turns from future R&D investments. 

It is not only leaders in the scientific and engineering fields, but policymakers as 
well in the Executive and Legislative Branches who recognize that we need better 
approaches for developing science policy, which is why the National Science and 
Technology Council established the Science of Science Policy Interagency Task 
Group. That task group’s roadmap characterized our current systems of measure- 
ment of the science and engineering enterprise as inadequate. There is enormous 
potential to do better. 

To begin to create a scientific, quantifiable measurement of the economic and so- 
cial impacts of our federal research investments, this Administration has initiated 
an innovative new program, STAR METRICS (Science and Technology in America’s 
Reinvestment — Measuring the EffecTs of Research on Innovation, Competitiveness 
and Science). This initiative is led by the National Institutes of Health and the Na- 
tional Science Foundation under the auspices of the White House Office of Science 
and Technology Policy. The goal is to develop a system that can be used to track 
the impact of federal science investments. I will return to the topic of STAR 
METRICS later in my testimony. 

1) The overall vision and strategy for research and education in the 
‘Science of Science and Innovation Policy. 

Federally funded basic and applied scientific research has had a significant im- 
pact on innovation, economic growth and America’s social well-being. We know this 
in the broad sense from numerous economic analyses but it is difficult to dis- 
entangle the impact of Federal investment versus private, state and industrial in- 
vestments. We bave little information about the impact of individual projects and 
programs, whether federally or privately funded. We have little information about 
the impact of science agencies. Thus, although determining which federally funded 
research projects 3deld solid results and which do not is a subject of high national 
interest, since American taxpayers invest more than $140 billion annually in re- 
search and development (R&D), there is little evidence to support such analysis. In 
short, although Congressional and Executive Branch policy decisions are strongly 
influenced by past practices or data trends that may be dated, or have limited rel- 
evance to today’s economic situation. A deeper understanding of the changing frame- 
work in which scientific and technical innovation occurs would help policymakers 
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decide how best to make and manage limited public R&D investments to exploit the 
most promising and important opportunities. 

The lack of analytical capacity in science policy is in sharp contrast to other policy 
fields that focus on workforce, health and education. Debate in these fields is in- 
formed by the rich availability of data, high quality analysis of the relative impact 
of different interventions and computational models that often allow for forward- 
looking analyses with impressive results. For example, in workforce policy, the eval- 
uation of the impact of distinct education and training programs has been trans- 
formed by careful attention to issues such as selection bias and the development of 
appropriate comparison groups. The analysis of data about geographic differences in 
health care costs and health care outcomes has featured prominently in guiding 
health policy debates. And education policy has moved from a “invest more money” 
and “launch a thousand pilot projects” imperative to a more systematic analysis of 
programs that actually work and that promote local and national reform efforts. 

Each of those efforts, however, has benefited from an understanding of the sys- 
tems that are being analyzed. In the case of science policy, no such agreement cur- 
rently exists. NSF’s Science of Science & Innovation Policy (SciSIP) program is de- 
signed to advance the scientific basis of science and innovation policy. 

Vision 

The principal goal is to advance the scientific basis of making science policy deci- 
sions, particularly those involving budgets, through the development of improved 
data collection, theoretical frameworks, computational models and new analytic 
tools. 

A major component of the SciSIP program is the funding of investigator initiated 
research. Through direct engagement of the federal policy community with the re- 
search community, it is hoped that future policy decisions can be informed by em- 
pirically validated hypotheses and informed judgment. Our aim, as noted in the pro- 
gram’s description, is to “engage researchers from all of the social, behavioral and 
economic sciences as well as those working in domain-specific applications such as 
chemistry, biology, physics, or nanotechnology in the study of science and innovation 
policy. The program welcomes proposals for individual or multi-investigator research 
projects, doctoral dissertation improvement awards, conferences, workshops, 
symposia, experimental research, data collection and dissemination, computer equip- 
ment and other instrumentation, and research experience for undergraduates. The 
program places a high priority on interdisciplinary research as well as international 
collaboration.” 

The program explicitly fosters a multi-level science (in addition to more obviously 
being an interdisciplinary science) that spans from the study of cognitive phe- 
nomena in individual scientists (e.g., the study of fixation, insight, reasoning, and 
decision making) to the study of whole industries and policies at the industry level. 
What makes the overall effort a potentially transformative effort is the support of 
research at multiple levels: industry level policies are only successful if it has indi- 
vidual-level effects (i.e., that engineers and scientists change), and individual-level 
effects are only important if they scale to produce industry-level differences. 

Another focus of the SciSIP program is the redesign of the surveys undertaken 
by NSF’s Science Resources Statistics, the federal statistical agency responsible for 
collecting and disseminating data on the U.S. science and engineering enterprise. 
The most visible activity has been the redesign of the Business R&D and Innovation 
Survey (BRDIS) which collects information from a nationally representative sample 
of about 40,000 companies, including companies in both manufacturing and non- 
manufacturing industries. This survey is the primary source of information on busi- 
ness, domestic and global R&D expenditures, and workforce. The new structure en- 
ables respondents to provide detailed data on the following: 

• How much is a company investing in its domestic and worldwide R&D rela- 
tionships, including R&D agreements, R&D “outsourcing,” and R&D paid for 
by others? 

• What is the strategic purpose of a company’s worldwide R&D activities and 
what are their technology applications? 

• What are the details of a company’s patenting, licensing, and technology 
transfer activities, and companies’ innovative activities? 

In addition, a limited number of questions are asked about activities related to 
new or improved products or processes. These are intended to serve as basis for col- 
lecting an expanded set of innovation metrics in the future. The results of this data 
collection are now being published as part of SRS’s ongoing reporting activity: 
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Strategy 

The focus of the program’s strategy has been to convince the academic community 
that the study of science policy is a worthwhile academic endeavor. This has taken 
three main forms. The first has been to engage in a substantial amount of outreach 
through presenting at professional workshops and conferences (an average of five 
or six a year), through supporting specific workshops on various science policy topics 
(two or three a year), through establishing a very active listserv (which has grown 
to over 720 members in less than two years) and through supporting a Science of 
Science Policy website (http:! I scienceofsciencepolicy.net). 

The second part of the strategy has been to invest in high quality research 
datasets. Good bricks need straw, and good research in an empirical field like 
science and innovation policy requires good data. Fields as disparate as bio- 
technology, geosciences, and astronomy have been transformed by both data and 
knowledge access. NSF hopes to similarly transform the study of science policy by 
improving science data. Such a transformation will occur in three ways. First, the 
scientific challenge is compelling: the way in which scientists create, disseminate 
and adopt knowledge in cyberspace is changing in new and exciting ways. Collabora- 
tions between computer scientists and social scientists, fostered by SciSIP, can cap- 
ture these activities using new cybertools. Second, new and exciting data attract 
new researchers to the field. This in turn attracts new graduate students, who see 
new ground being broken and exciting opportunities for research. Finally, we aim 
to actively engage the federal science policy community through a variety of work- 
shops, as well as direct engagement through the Science of Science Policy Inter- 
agency Group. 

The program has made a total of 99 awards: 19 in 2007, 23 in 2008, 31 in 2009, 
and 26 in 2010. The program began to accept doctoral dissertation proposals in 
2010; five of those were funded. The success rate for standard proposals is currently 
about 25%; higher for doctoral dissertation proposals. A total of 182 principal inves- 
tigators have been supported — of those 147 are scientists from Social, Behavioral 
and Economic Science domains and the balance are from areas as diverse as Com- 
puter and Information Sciences, Education, Physics, Biology and Law. 

Results 

The program is beginning to achieve some of its ambitious goals. A SciSIP Prin- 
cipal Investigator (PI), who is a Business School Dean at a university with a strong 
focus on publicly-funded research, has noted “I know full well that this new program 
provides unique grant opportunities for faculty members in management, informa- 
tion systems, and other fields of business administration. He cites the following 
from his personal experience “ ... in the field of business research, and business 
management, the Science of Science & Innovation Policy papers are featured in 
some of the best sessions at the Academy of Management Meetings. This innovative 
program has sparked considerable interest in public policy among management 
scholars, and particularly in business schools. The impact of the research you are 
funding struck home when I read the latest issue of BizEd, the magazine of The 
Association to Advance Collegiate Schools of Business (AACSB, an association of 
educational institutions, businesses, and other organizations devoted to the advance- 
ment of higher education in management education. It is also the premier accred- 
iting agency of collegiate business schools and accounting programs worldwide. The 
research you have funded was prominently featured in their magazine, which is cir- 
culated to thousands of business schools worldwide.” 

Additionally, the impact of the SciSIP program has influenced several National 
Research Council studies, and thus impacted public policy with respect to tech- 
nology commercialization and academic and public sector entrepreneurship. One is 
the Congressionally-mandated evaluation of the Small Business Innovation Re- 
search Program. Another is a committee entitled “Best Practices in National Innova- 
tion Programs for Flexible Electronics” and a third is “Management of University 
Intellectual Property: Lessons from a Generation of Experience, Research, and Dia- 
logue” 

In another example, a major part of the science and innovation policy debate has 
been the role of R&D and research tax credits whose budgetary cost is about $15 
billion each year i. The obvious policy question is how effective are these tax credits 
in stimulating innovation? SciSIP funded Pis have examined changes in R&D tax 
credit generosity across countries and US states over time to evaluate business 
firms’ response. They estimate that for every $1 of tax credits firms spend about 
$1 more on R&D. However, the research also extends to the impact of firms’ re- 


http:! I www.ncseonline.org I NLE I CRSreports I OSAug / RL31181.pdf, CRS-3 
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sponse to the uncertainty about the duration of the federal Research and Experi- 
mentation (R&E) tcix credit, which is not currently permanent and in fact expired 
at the end of 2009. The uncertainty about renewal has offsetting effects — one is to 
increase short-term expenditures because firms think they need to do R&D now to 
get the credit. The reverse is to reduce overall R&D expenditures since uncertainty 
is detrimental to the expected payoffs from long-term investments such as funda- 
mental R&D. The sign of the net effect is an empirical question, and again some- 
thing the SciSIP PI has been working on — he finds a strong negative effect of uncer- 
tainty on general investment and employment, and is currently extending this work 
to R&D. The same PI presented in September 2009 to the Federal Reserve Bank 
Board of Governors, including Governor Bernanke; the Fed was tr 3 dng to under- 
stand why the IT “productivity miracle”, which was a major driver of US economic 
growth in the late 1990s, has slowed down by the late 2000s. One possible reason 
is that better use of IT is associated with organizational change, and the rate of or- 
ganizational change has potentially slowed down; a major SciSIP-funded grant sup- 
ports collecting a large national survey to try to examine why and how that change 
has occurred. 

We can also learn from history. Another SciSIP PI has looked at two case studies 
in depth: the invention of the airplane and Edison’s invention of the electric light. 
In both cases, the invention took a long period of time — 110 years and 80 years, re- 
spectively. In both cases even the earliest attempts were based on many years [of 
work on mathematics and technology and hundreds of years of work of science. To 
illustrate. Sir Humphry Davy first demonstrated incandescence of materials in 
1808. His work drew on the Voltaic pile (battery) invented in 1800, the Leyden jar 
developed in 1744, and carbon produced as charcoal during the Roman Empire no 
later than 25 A.D. Leyden jars depended on work by the ancient Greeks in 600 B.C. 
Thus, the foundation of the science behind electric light dates back 2400 years be- 
fore incandescence, after which it took 80 more years of R&D to develop an effective 
electric light. The airplane also has a similarly long foundational period and dura- 
tion of invention. In looking at various inventions, research has shown that there 
are several different weak methods but also some powerful strategies that vastly 
speed things along. Edison succeeded simply because he had enormous resources 
(the Edison Electric Light Company was capitalized at $300,000 — about $30 million 
today. The Wright Brothers were far more efficient at developing the airplane than 
Edison was in developing the electric light. 

How is the NSF fostering collaboration between social and behavioral sci- 
entists and researchers from other disciplines, including computer 
scientists, engineers and physical scientists in science and tech- 
nology policy research? 

This is being done in a number of ways: through the program call, through work- 
shops, and through successful and visible interdisciplinary projects. 

Program Description 

The SciSIP program explicitly encourages interdisciplinary cooperation in the pro- 
gram description. In particular, the program states 

“The SciSIP program invites the participation of researchers from all of the so- 
cial, behavioral and economic sciences as well as those working in domain-spe- 
cific applications such as chemistry, biology, physics, or nanotechnology. The 
program welcomes proposals for individual or multi-investigator research 
projects, doctoral dissertation improvement awards, conferences, workshops, 
symposia, experimental research, data collection and dissemination, computer 
equipment and other instrumentation, and research experience for undergradu- 
ates. The program places a high priority on interdisciplinary research as well 
as international collaboration.” 

Workshops 

Most of the workshops that have been hosted have been explicitly interdiscipli- 
nary in nature, bringing together domain scientists and social, behavioral and eco- 
nomic scientists, and have resulted in calls for proposals (called Dear Colleague Let- 
ters) supported lay multiple NSF programs. 

Examples include: 

• A two-day workshop to advance the scientific study of federally funded cen- 
ters and institutes as key elements in the innovation ecosystem. The work- 
shop brought together engineers and natural, physical, and social scientists 
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to address central questions relating to the role of NSF-funded centers and 
institutes in science and innovation policy. 

• Two separate workshops studying innovation in organizations. One of these, 
hosted by the Conference Board and supported by four Social, Behavioral and 
Economic (SBE) Sciences and three Computer and Information Science and 
Engineering (CISE) programs, was attended by computer scientists, SBE sci- 
entists and representatives from the business community to examine the po- 
tential for cyber data to better inform our understanding of innovation. A sec- 
ond conference brought together 20 leading computer scientists (from the 
fields of data management, data mining, security/privacy, social networks) 
and social/organizational scientists (that included economists, sociologists, 
psychologists, anthropologists) to identify emerging major challenges in the 
collection and use of confidential data collection for the study of innovation 
in organizations. SciSIP led the reusulting development of a Dear Colleague 
Letter whose purposes was to gather and create new Cyber-enabled data on 
innovation in organizations, supported by six SBE and four CISE programs 
as well as the Office of Cyber Infrastructure. 

• A workshop in conjunction with the NSF’s Chemistry Division that examined 
the impact of science R&D in the United States, focusing on chemical sciences 
and related industries. This led to a Dear Colleague Letter from SciSIP and 
the Chemistry Division reaching out to the chemistry and the social science 
communities advising them of funding opportunities related to assessing and 
enhancing the impact of R&D in the chemical sciences in the United States. 

• An interdisciplinary workshop which examined the potential for new visual- 
ization tools to track the impact of investments in science. These possibilities 
include tracing the impact of basic research on innovation, examining the 
changing structure of scientific disciplines, studying the role of social net- 
works in the dispersion of scientific innovations as well as making compari- 
sons of how the U.S. compares internationally in science. That workshop 
brought together researchers from a broad range of disciplines to examine 
such key questions, and to engage the federal science community in a discus- 
sion about whether and how the tools could be used in the federal context. 

• Three workshops have directly engaged CISE and SBE researchers in enhanc- 
ing NSF’s ability to describe its research portfolio. The SciSIP program 
worked with the CISE directorate to form an Advisory subcommittee to pro- 
vide advice on approaches to improving the way NSF interacts with its pro- 
posal and award portfolio. Although NSF staff still rely on traditional meth- 
ods to do their jobs, such methods are becoming less practical given the rap- 
idly changing nature of science, the increased recognition of the importance 
of funding interdisciplinary and potentially transformative research, and the 
significant increase in the number of proposals submitted. Individuals with 
research expertise in machine learning, data mining, information visualiza- 
tion, human-centered computing, science policy, and visual analytics were re- 
cruited for this effort. Nine teams were put together and charged with pro- 
viding advice to NSF on identifying and demonstrating techniques and tools 
that characterize a specific set of proposal and award portfolios. Their report, 
in turn, will advise NSF on how to better structure existing data, improve use 
of existing machine learning, analysis, and visualization techniques to com- 
plement human expertise and better characterize its programmatic data. The 
results should help NSF identify tools that will help fulfill its mission includ- 
ing identifying broader impacts, as well as funding transformative and inter- 
disciplinary research. NSF has also engaged program managers across the 
federal government so that our collective approaches can inform not only us, 
but other science agencies. 

• A workshop responding to Congressman Holt’s request for better ways to 
measure the economic impact of federal research investments. SciSIP, to- 
gether with NIH and other agencies, is supporting the National Academy of 
Sciences’ Board on Science, Technology, and Economic Policy (STEP) and 
Committee on Science, Engineering, and Public Policy (COSEPUP) 2011 
workshop on science measurement. This workshop is aimed at discussing new 
methodologies and metrics that can be developed and utilized for assessing 
returns on research across a wide range of fields (biomedical, information 
technology, energy, and other biological and physical sciences, etc.), while 
using background papers that review the methodologies used to date as a 
starting point. 
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As one SciSIP PI has noted, “SciSIP.. creates a domain around which researchers 
from a variety of disciplines — biology and physics and economics as well as informa- 
tion science and public policy — can coalesce to pursue research topics in this domain 
for their own sake, rather than in the interstices of other projects in their home dis- 
ciplines. As such, it acts as an attractor for top researchers across the natural and 
social sciences, allowing them to pursue their interests in SciSP topics” 

Successful Examples 

There are a number of examples of the fruits of these activities. For example, 
SciSIP funded research supports a University of Michigan research team consisting 
of a sociologist, a bioethicist specializing in informed consent and stem cell regula- 
tion, a bioethicist trained as a molecular biologist who is working on cell banking, 
and a post-doc in stem cell biology. The combination is a powerful one as it matches 
expertise with social scientific data and analysis methods, with deep knowledge 
about both the policy and the science. 

Similarly, the interdisciplinary work of two SciSIP Pis has helped developed new 
metrics of the transmission of knowledge. These metrics go beyond citation metrics 
to usage metrics and help us better understand the impact that federal investment 
in research is having on research results. By mapping the structure of science and 
looking at how this structure changes over time, we can see the shifting landscape 
of scientific collaboration and understand the new emerging disciplines. That will 
enable us to to anticipate these changes and properly target research funding to 
new and vibrant areas. For instance, their work provides a striking example of the 
emergence of neuroscience over the past decade — changing from an interdisciplinary 
specialty to a large and influential stand alone discipline on a par with physics, 
chemistry, or molecular biology. 

How is NSF fostering the development of seience and teehnology policy de- 
gree programs and courses of study at colleges and universities? 
What is the current scope and level of support for such programs? 

As with many NSF programs, the SciSIP program explicitly encourages submis- 
sions that support graduate student development. While there is no direct targeting 
of funds to policy programs, SciSIP supported 28 researchers from science and tech- 
nology policy programs. In an example of the type of support that has been provided 
to expand the course of study, over 250 undergraduate students from Economics (be- 
havioral economics). Cognitive Science, Electrical and Computer Engineering, and 
Industrial Engineering have participated in a project at Purdue University, which 
is an interdisciplinary collaboration linking social scientists and computer scientists 
and engineers. 

A further example is the work done by Marcus Ynalvez at Texas A&M Inter- 
national University, which has the explicit goal of mentoring TAMIU Graduate Stu- 
dents (Students from Historically Underrepresented Populations): The hands-on 
training and mentoring of TAMIU graduate students represents an attempt to en- 
gage Hispanic students in international scientific research activities with the inten- 
tion of introducing them to the possibilities of developing professional careers in 
science and technology. These students are currently gathering, synthesizing and re- 
viewing literature materials for the project’s manuscripts, publications, and reports. 
With the data from the Japan, Singapore, and Taiwan surveys, these students will 
be analyzing data using a number of statistical software such as: Statistical Pack- 
ages for the Social Sciences (SPSS), STATA, and Statistical Analysis System (SAS). 
They will learn how to interpret statistical results associated with the family of gen- 
eralized linear regression models, namely: linear, logistic, and negative binomial re- 
gression models, analysis of variance, and path analysis. Not only have the TAMIU 
graduates gained actual research experience, they have also developed professional 
relationships with students and professors from the prestigious National University 
of Singapore 

How is NSF encouraging a community of practice in science of science pol- 
icy and the dissemination of policy to policy makers 

A major avenue has been the linkage with the Science of Science Policy Inter- 
agency group, which is discussed in more detail below. In addition, the listserv and 
the website have been very important dissemination vehicles. 

However, the most important vehicle has been two PI workshops with the explicit 
goal of fostering further collaboration among the Pis actively engaged in the study 
of Science of Science & Innovation Policy and the link to the federal community. 
The 2009 workshop had three overarching goals: 
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• to provide NSF with an early opportunity to organize a collegial discussion 
of work in progress under SciSIP’s two rounds of awards well before this work 
will begin to appear in professional forums and publications; 

• to begin to develop from among the purposefully diverse set of disciplinary 
perspectives reflected in SciSIP’s two solicitations and subsequent awards, a 
“community of experts across academic institutions and disciplines focused on 
SciSIP;” and 

• To identify new areas of emphasis for support in future SciSIP solicitations. 

The 2010 workshop, scheduled for October 19, 2010 seeks to focus on two objec- 
tives that flow from the National Science and Technology Council’s 2008 report: The 
Science of Science Policy: A Federal Research Roadmap. The first task, as called for 
in the Roadmap report, is “to advance the scientific basis of science policy so that 
limited Federal resources are invested wisely.” The second is to build a “community 
of practice” between Federal science and technology policymakers and researchers 
engaged in the development of new theories, tools of analysis, and methods for col- 
lecting and analyzing data. 

This October 2010 workshop will consist of brief presentations by a number of 
SciSIP grantees who have been invited to participate via a competitive peer review 
of abstracts previously submitted based on their ongoing research. These presen- 
tations will be followed by roundtable discussions led by federal policymakers who 
will comment on the relevance of the research, followed then by open discussions 
among all participants. A networking session will be scheduled at the close of the 
formal sessions to allow for continued discussion. 

2) As a Co-chair of the Science of Science Policy Interagency Group under 
the NSTC please briefly describe the work of that group and how 
the various federal science agencies are collaborating on the de- 
velopment and implementation of science of science policy tools to 
improve the management and efficiency of their R&D portfolios 
and other science and technology related programs. 

In 2006, the National Science and Technology Committee’s Subcommittee on So- 
cial, Behavioral and Economic Sciences (SBE) established an Interagency Task 
Group on Science of Science Policy (ITG) to serve as part of the internal deliberative 
process of the Subcommittee. In 2008, this group developed and published The 
Science of Science Policy: A Federal Research Roadmap which outlined the Federal 
efforts necessary for the long-term development of a science of science policy, and 
presented this Roadmap to the SoSP Community in a workshop held in December 
2008. The ITG’s subsequent work has been guided by the questions outlined in the 
Roadmap and the action steps developed at the workshop. 

The development of the STAR METRICS (Science and Technology for America’s 
Reinvestment: Measuring the EffecT of Research on Innovation, Competitiveness 
and Science) program is the number one priority of the interagency group. The ini- 
tiative is a multi-agency venture led by the National Institutes of Health, the Na- 
tional Science Foundation (NSF), and the White House Office of Science and Tech- 
nology Policy (OSTP). 

Another major activity is sponsoring a series of workshops to bring the science 
agencies together to share what is already established in the field, identify gap 
areas and outline steps forward for the creation of better tools, methods, and data 
infrastructure. 

The first of these workshops was held in October, 2009 to delve into the issues 
surrounding performance management of federal research and development port- 
folios. The focus was on sharing current practices in federal R&D prioritization, 
management, and evaluation. Over 200 agency representatives attended. The con- 
ference featured 27 speakers and panelists, representing 20 federal agencies, offices, 
and institutions, and over 30 poster presenters, representing more than 25 agencies 
and institutions. Topics that were discussed included: 

• Methods to set federal research priorities and strategic directions; 

• The use of metrics to improve federal R&D efficiency; and 

• Ways in which research evaluations can inform current and future R&D deci- 
sions. 

It addressed the following key questions: 

• How do federal science and technology agencies systematically identify and 
prioritize research and development alternatives? How can these processes be 
strengthened? 
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• How can research-performance metrics be used to improve research effi- 
ciency? How can these metrics be improved? 

• How do research-performance evaluations inform and improve R&D invest- 
ment decisions? How can these feedback loops be reinforced? 

While the 2009 workshop developed a dialogue within the federal science policy 
community, the ITG has a workshop planned for December 2010 that engages the 
federal community with the academic community in advancing the “Science of 
Science Measurement”. The first goal is to create a dialogue between the Federal 
S&T agencies and the research community about relevant models, tools, and data 
that advance scientific measurement in key areas of national S&T interest. The sec- 
ond objective is to identify a joint Science of Science Policy (SoSP) research agenda 
for the Federal S&T agencies and the research community. The workshop has four 
modules intended to advance measurement in: 1) Economic benefits; 2) Social, 
health and environmental benefits; 3) S&T workforce development; and 4) Tech- 
nology development and deployment. Four academic researchers will be presenting 
in each module, with a rapporteur synthesizing the presentation at the end of each 
module. 

The audience will be primarily science policy practitioners from the Federal agen- 
cies who are interested in very practical issues, such as: getting new ideas about 
how to manage their portfolios in a more scientific manner; developing performance 
and outcomes metrics; measuring the return on investment; and using science to 
identify emerging trends in the U.S. scientific enterprise. 

Another activity has been the establishment of a website to provide information 
on best practices to Federal and non-Federal agencies. The website (http:! / 
scienceofsciencepolicy.net) was launched in January 2010, and has become a model 
for other interagency groups (including the Forensic Science interagency group). The 
web site serves as a repository for data, documents, research papers, and commu- 
nication tools for the communities of users. , The site receives over 2,000 hits a 
month. The associated Listserv is the highest visibility listserv in science policy, and 
has over 720 members. 

The interagency group meets monthly, and has active participation by over 15 
agencies. It is actively providing input to the Center of Excellence on Science Policy 
being established by the State Department in the Middle East. 

3) Please provide a brief description and update on the status of the OSTP 
led project on science metrics, known as STAR METRICS, includ- 
ing a description of international engagement and interest in this 
effort 

The STAR METRICS project is a federal and university partnership to document 
the outcomes of science investments to the public. The benefits of STAR METRICS 
are that a common empirical infrastructure will be available to all recipients of fed- 
eral funding and science agencies to quickly respond to State, Congressional and 
0MB requests. It is critical that this effort takes a bottom up approach that is do- 
main specific, generalizable and replicable. 

Currently, the project is structured in two phases: 

- Phase I: The development of uniform, auditable and standardized measures 
of the initial impact of ARRA and base budget science spending on job cre- 
ation. 

- Phase II: The development of broader measures of the impact of federal 
science investment, grouped in four broad categories: 

• Scientific knowledge (such as publications and citations) and, later, 

• Social outcomes (such as health and environment), 

• Economic growth (through patents, firm start ups and other measures), 

• Workforce outcomes (through student mobility and employment), 

Phase I of the STAR METRICS project began in earnest in March of 2010 with 
funds formally designated for the project. The participation agreement was signed 
in May 2010, and a press release was issued by the three lead agencies: NIH, NSF 
and OSTP 2. As noted in that press release: 

“A new initiative promises to monitor the impact of federal science investments 

on employment, knowledge generation, and health outcomes. The initiative — 

Science and Technology for America’s Reinvestment: Measuring the EffecT of 


2 http: ! I WWW. whitehouse.gov ! sites i default ! files I microsites I ostp I STAR%20METRICS 
%20FINAL.pdf 
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Research on Innovation, Competitiveness and Science, or STAR METRICS — is 
a multi-agency venture led by the National Institutes of Health, the National 
Science Foundation (NSF), and the White House Office of Science and Tech- 
nology Policy (OSTP).” 

In Phase I, through a highly automated process, with essentially no burden on 
scientists and minimal burden for administrators, STAR METRICS collects longitu- 
dinal employment data from the participating institutions to be able to assess the 
number of jobs created or retained (or lost) through federal funding support. The 
system is set up such that all jobs will be captured and not just principal investiga- 
tors and co-principal investigator. In addition, in Phase I, STAR METRICS can pro- 
vide estimates of jobs supported through facilities and administration (F&A) costs 
and through various procurement activities in the institutions. 

STAR METRICS will also help the Federal government document the value of its 
investments in research and development, to a degree not previously possible. To- 
gether, NSF and NIH have agreed to provide $1 million in funding a year for the 
next five years. 

More agencies are joining the STAR METRICS consortium. While meetings of the 
Consortium are convened by OSTP, the lead agency is NIH, which is hosting the 
data infrastructure. The official STAR METRICS website will be available Sep- 
tember 30 2010. NSF is providing key leadership in engaging the scientific commu- 
nity, particularly through the SciSIP program. 

phase II of the project expands the data infrastructure to incorporate the broader 
impact of science investments on scientific, social, economic and workforce outcomes. 
In keeping with the bottom up approach of the program, STAR METRICS is begin- 
ning a formal set of consultations with the scientific community to understand what 
data elements and what metrics the community would find useful to find in STAR 
METRICS. The first of these will occur October 22, 2010, with a meeting with Vice 
Presidents for Research of interested institutions. Other meetings will follow with 
research agencies and other interested groups. 

In a very short period of time since formalizing the project, over 100 research- 
intensive universities, mostly from the Federal Demonstration Partnership (FDP), 
have expressed interest in participating in STAR METRICS; about 20 are contrib- 
uting data. Universities have expressed enthusiasm and support for the project. 

Science is fundamentally an international endeavor. And so must be its evalua- 
tion. In fact, there has been substantial international interest. Members of the 
STAR METRICS team have provided information or directly briefed Brazilian and 
Japanese science and technology agencies. The State Department is actively inter- 
ested in learning about the program to advance the science of science policy in the 
Middle East. 

Our most active international counterpart, however, is the European Union. A 
major presentation was given to the European Parliament in ApriU. A joint EUAJS 
conference has been proposed for March 2011 in the Rockefeller Foundation’s 
Bellagio Center. The goal is to produce a roadmap that will outline a path for cre- 
ating a US/European collaboration in developing a common theoretical and empir- 
ical infrastructure to describe and assess the outcomes of science investments. To 
achieve this, it will bring together key European and US science policy experts and 
makers, administrators and academic researchers. The group is carefully chosen to 
consist of the key players from the US side who have the experience in developing 
such an infrastructure in the US. The European attendees will consist of individuals 
who have both the deep understanding of the issues and the ability to effect change 
in Europe in a collaborative framework with the US. 

The outcomes will include a roadmap that represents a combined effort to build 
on and extend existing efforts in both regions: notably the US investment in the 
STAR METRICS program and the European efforts to build better assessments for 
their investments. It is hoped that the roadmap will have the same success that the 
Science of Science Policy Interagency Roadmap had in the United States and that 
in the EU the road map will be the basis for including assessment measures in fu- 
ture legislation implementing science programs. 

Conclusion 

The NSF’s Science of Science and Innovation Policy program, the NSTC’s Inter- 
agency SOSP ITG, along with STAR METRICS, represent the first efforts to con- 
struct a scientific framework that is supported by multiple agencies and multiple 
institutions — all jointly engaged. It represents a true bottom up approach to pro- 


3 http:! I www.euractiv.CQm I en i science i eu-looks-to-us-model-for-measuring-rd-impact-news- 
448950 
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viding an evidence basis for U.S. science policy. Its success is important for decision- 
makers: in a nutshell, you can’t manage what you can’t measure and what you 
measure is what you get. 

NSF’s innovative Science of Science and Innovation Policy program, and STAR 
METRICS, can help all of us do a better job in explaining this essential symbiosis. 

This concludes my testimony, Mr. Chairman. I look forward to answering any 
questions you or Members may have. 

Biography for Julia I. Lane 

Dr. Julia I. Lane is the Program Director of the Science of Science & Innovation 
Policy program at the National Science Foundation. Her previous jobs included Sen- 
ior Vice President and Director, Economics Department at NORCAJniversity of Chi- 
cago, Director of the Employment Dynamics Program at the Urban Institute, Senior 
Research Fellow at the U.S. Census Bureau and Assistant, Associate and Full Pro- 
fessor at American University. 

Julia has published over 60 articles in leading economics journals, and authored 
or edited six hooks. She became an American Statistical Association Fellow in 2009. 
She has heen the recipient of over $20 million in grants; from foundations such as 
the National Science Foundation, the Sloan Foundation, the MacArthur Foundation, 
the Russell Sage Foundation, the Spencer Foundation, the National Institute of 
Health; from government agencies such as the Departments of Commerce, Labor, 
and Health and Human Services in the U.S., the ESRC in the U.K., and the Depart- 
ment of Labour and Statistics New Zealand in New Zealand, as well as from inter- 
national organizations such as the World Bank. 

She has organized over 30 national and international conferences, received several 
national awards, given keynote speeches all over the world, and serves on a number 
of national and international advisory hoards. She is one of the founders of the 
LEHD program at the Census Bureau, which is the first large scale linked em- 
ployer-employee dataset in the United States. A native of England who grew up in 
New Zealand, Julia has worked in a variety of countries, including Australia, Ger- 
many, Malaysia, Madagascar, Mexico, Morocco, Namibia, Sweden, and Tunisia. 

Her undergraduate degree was in Economics with a minor in Japanese from 
Massey University in New Zealand; her M.A. in Statistics and Ph.D. in Economics 
are from the University of Missouri in Columbia. She is fluent in Swedish and Ger- 
man and speaks conversational French. 

Chairman LiPiNSKl. Thank you, Dr. Lane. Dr. Sarewitz. 

STATEMENT OF DANIEL SAREWITZ, CO-DIRECTOR OF THE 

CONSORTIUM FOR SCIENCE, POLICY & OUTCOMES AND 

PROFESSOR OF SCIENCE AND SOCIETY, ARIZONA STATE 

UNIVERSITY 

Dr. Sarewitz. Thank you, Chairman Lipinski and Ranking Mem- 
ber Ehlers. I very much appreciate the invitation and the oppor- 
tunity to testify. So, my name is Daniel Sarewitz. I am a Professor 
of Science and Society at Arizona State University where I Co-Di- 
rect the Consortium for Science Policy and Outcomes, which works 
to understand and improve the linkages between science and tech- 
nology and social outcomes. We are located on ASU’s Tempo cam- 
pus. We also have a location here in DC. We are a highly inter- 
disciplinary and collaborative organization involving researchers at 
dozens of other institutions. We are also fortunate to receive gen- 
erous grant funding from NSF, including from the Science of 
Science and Innovation Policy Program, so I declare my vested in- 
terest in the outcomes of this hearing. 

I would like to make three brief points in support of my over-ex- 
tensive written testimony. The first is about the importance of the 
SciSIP Program itself. With shrinking discretionary budgets, vi- 
brant economic competitors, and daunting challenges to our well- 
being, the Nation needs effective tools for making better decisions 
about how to design, assess, and set priorities for our science and 
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innovation enterprise. For the most part, we lack these tools, as we 
have already heard. As former Presidential Science Advisor Jack 
Marburger said in 2005, “the nascent field of social Science of 
Science Policy needs to grow up, and quickly.” 

With modest resources, SciSIP is mobilizing a community of re- 
searchers to focus on the complex problem of how to bring the most 
out of our public investment in R&D. SciSIP reacted quickly to sup- 
port research assessing the impacts of stimulus funding for R&D, 
and is beginning with NIH to take on the incredibly complex prob- 
lem of evaluating what the nation gets for its enormous investment 
in bio-medical R&D. These are really difficult challenges and it is 
hard to see how this committee and others at the helm of the R&D 
enterprise can guide it effectively in the absence of such efforts. 

A second point is that outputs are not outcomes. And SciSIP 
needs to focus on outcomes. Outputs are immediate products of 
R&D like publications, and patents, and Ph.D’s. Outcomes are 
what people care about, not just economic growth, but of course 
economic growth, but also secure and affordable food supplies and 
energy supplies, high quality public health, a clean environment, 
expanding job opportunities and strong national defense. The 40- 
year war on cancer has yielded the output of remarkable new sci- 
entific knowledge yet very modest gains in public health outcomes 
despite the tens of billions spent. Thirty years of energy R&D out- 
put have done little to advance the outcome of reducing our vulner- 
ability to energy-based threats to security, economy, and environ- 
ment. Research on science and innovation policy to date has given 
us a pretty good idea how to design and assess science policies to 
advance outputs. But we still have a lot to learn about how to im- 
plement and assess successful outcome-based science and innova- 
tion policies. 

My final point is that research on outcome-based science and in- 
novation policies and the use of such research by decision makers 
are not separate problems. While the SciSIP Program is commend- 
ably serious about disseminating its research and its results to pol- 
icy makers, the dissemination problem is also structural. That is it 
is built into the way we organize much research including SciSIP, 
and its great strength in supporting bottom-up inquiry on funda- 
mental problems is also a weakness when there is an urgent need 
for new knowledge, the need that Dr. Marburger pointed out. Such 
cases require close ties between those who do research and decision 
makers who might use research results. We already heard from 
Julia Lane about her efforts to create those ties. Now on the one 
hand, these ties allow researchers to understand the needs of deci- 
sion makers and to recognize the types of information that will be 
both usable and used. But at the same time, close ties allow deci- 
sion makers to understand what research can and cannot do for 
them. Such mutual understanding breeds trust and value, and usa- 
ble science. 

There are many examples of federal programs that link research 
performance and research use, including USDA’s Agricultural Ex- 
tension Service, the USGS Earthquake Hazards Program, and 
NOAA’s Regional Integrated Sciences and Assessments Program. 
Similarly, DARPA is justifiably well regarded for its capacity to 
connect the technology needs of DOD to research groups in aca- 



26 


demia and the private sector. These and other examples are dis- 
cussed in the handbook “Usable Science”, which I just happened to 
have brought along with me, which summarizes the results of 
CSPO’s [Consortium for Science, Policy & Outcomes] five year NSF 
Decision Making Under Uncertainty project, carried out jointly 
with researchers of the University of Colorado. 

These lessons can be applied to SciSIP. Let me mention three 
possibilities. First, NSF could sponsor one or more large centers for 
SciSIP research, education, and outreach with a core requirement 
to build strong, ongoing collaborative links between researchers 
and science policy decision makers. Second, NSF could work with 
mission-oriented R&D agencies to integrate SciSIP activities into a 
range of existing outcome-oriented programs. Third, NSF could re- 
quire all of its center-scale awardees, such as Science and Tech- 
nology Centers and Engineering Research Centers, to be designed 
from the outset to include integrated SciSIP components. 

Through these sorts of approaches, SciSIP could enhance its ca- 
pacity to produce usable knowledge for the near to medium term 
and help accelerate a convergence between science and innovation 
policy research and policy decisions across a range of R&D outcome 
priorities. Thank you for your attention. I look forward to dis- 
cussing these issues more. 

[The prepared statement of Dr. Sarewitz follows:] 

Prepared Statement of Daniel Sarewitz 

Mr. Chairman, Members of the Committee, thank you for inviting me to testify 
today. My name is Daniel Sarewitz, and I am co-founder and co-director of the Con- 
sortium for Science, Policy, and Outcomes at Arizona State University, as well as 
Professor of Science and Society at ASU. My formal training was in geosciences, but 
for more than 20 years I have worked in science and technology policy, first as a 
AAAS Congressional Science Fellow and then a staffer on this Committee, working 
for Chairman George E. Brown, Jr., and more recently as an academic, at Columbia 
University and now at ASU. So I’m very pleased to return to the place that 
launched me on a new and incredibly interesting and exciting career path and intel- 
lectual journey, and honored that you have asked for my input to the Committee’s 
deliberations on the status of the science of science and innovation policy. 

Introduction: Input-Output Science and Innovation Policy 

Most people agree that government support of research and development is an es- 
sential foundation of today’s complex, knowledge-based, high technology society. Yet 
the problem of how to make the most out of the nation’s investment in R&D re- 
mains amazingly poorly understood. This problem has been actively debated in Con- 
gress since World War II. In the interim the annual public investment in R&D has 
grown from a few tens of millions of dollars to about 140 billion dollars. Yet, 
throughout this period of remarkable growth — and, I should say, remarkable bipar- 
tisan support for such growth, exemplified by this Committee — the basic principles, 
terms of debate, and policy tools for guiding investment and measuring its effects 
have changed remarkably little. 

For more than sixty years, the core of science policy has been the belief that more 
money for R&D translates into more benefits for the nation. Science policy has, 
above all else, been science budget policy. The capacity of the nation to solve prob- 
lems related to science and technology has been measured by the incremental 
growth of the R&D budget. The idea that the size of the R&D budget is a measure 
of the social value of science and technology remains the bedrock of science policy. 

Three other powerful beliefs have dominated science policy decision making. The 
first is that research becomes valuable for society as part of a linear progression 
starting with basic discovery and leading to application, either in the form of tech- 
nological innovation, or information to inform decision making. The second, related 
belief is that there is a clear distinction between research activities aimed at cre- 
ating new knowledge, and research aimed at appljdng that knowledge to solving 
problems. The third belief is that scientific excellence, as defined and assessed by 
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scientists themselves, typically through the peer review process, is the best measure 
of the potential value of science for society. 

The result of these beliefs has been a national R&D enterprise that is largely un- 
derstood and discussed in terms of simple inputs — how much money is being spent 
on which type of science? — and simple outputs — how much scientific knowledge is 
being produced? That this simple input-output way of understanding science and 
technology policy led to the world’s largest and most productive R&D enterprise is, 
however, much more of a happy historical accident than an endorsement of this way 
of looking at R&D policy. 

Coming out of World War II, the U.S. simply had no serious scientific or economic 
competitors, so we had a huge head start that only began to be seriously eroded in 
the 1980s. Moreover, the U.S. R&D enterprise as a whole was — and still is — so 
much bigger than that of any other nation that simply as a function of scale it 
could — and still does — outperform everyone else. An additional crucial point is that 
by far the dominant player in translating the public R&D investment into tangible 
societal outcomes was the Department of Defense. The core of DOD’s approach was 
the cultivation of very powerful linkages between high-tech private sector firms, re- 
search universities, and the DoD itself, an arrangement that was responsible for 
creating most of the important technological systems that undergird our society and 
our economy today. 

I present this thumbnail sketch to explain how we have arrived at the situation 
in which we find ourselves today. The limits of the post-War input-output approach, 
as I have said, became increasingly clear starting in the 1980s, with the rise of seri- 
ous economic and technological competitors, especially in east Asia; with the end of 
the Cold War, and the decline of DoD’s catalytic role in civilian technological innova- 
tion; and with the increasing awareness of an array of social challenges that seemed 
to demand scientific and technological solutions — from cancer and emerging infec- 
tious diseases to energy security and environmental quality. Yet if one looks at the 
endless series of reports over the past decades sounding the alarm bells about the 
nation’s science and technology enterprise, one finds the problem still discussed pre- 
dominantly in terms of the same old input-output measures: how much are we 
spending, how many scientists are we producing, how many publications or patents 
are issued, and how do these input-output numbers compare to our economic com- 
petitors? 

The problem with the input-output model is that it can’t tell us very much about 
what actually matters: how the size, organization, and productivity of the R&D en- 
terprise itself relates to the achievement of the societal outcomes that we desire and 
expect. Because pretty much everyone assumed that these outcomes flowed auto- 
matically from the R&D enterprise, as long as it was big and scientifically produc- 
tive, there seemed to be no reason to worry about how the enterprise worked. These 
assumptions put a damper on research, as well as debate, about the complex rela- 
tions between scientific advance, technological innovation, and the well-being of soci- 
ety. Why try to understand these issues if the only thing that really mattered was 
the size of the budget? 

But in an era of constrained resources and mounting challenges to our well-being, 
the limits of the input-output approach have become impossible to ignore. We can- 
not ignore them because we need to make difficult choices about how to allocate 
scarce resources. We also cannot ignore them because we are faced with strong 
prima fascia evidence that the input-output approach is leading to significant 
science and innovation policy failures. For example, the National Institutes of 
Health’s forty year War on Cancer has yielded remarkable new scientific knowledge, 
yet remarkably modest public health benefits for the tens of billions spent. The dev- 
astation of New Orleans by Hurricane Katrina occurred despite the existence of 
comprehensive scientific knowledge about the inevitability and precise consequences 
of such an event. Thirty years of energy R&D has left the nation no less vulnerable 
to energy-based security, economic, and environmental threats than it was when the 
Department of Energy was created. These are not input-output problems, but they 
are science and innovation policy problems. 

In 1992, this Committee issued a brief “Chairman’s Report” entitled the “Report 
of the Task Force on the Health of Research,” which pointed at the need to re-think 
basic assumptions about science and innovation policy. (As a Committee staffer at 
the time, I was privileged to be one of the members of that Task Force.) While there 
certainly were, at that time, small pockets of academic scholarship on the links be- 
tween science policy and societal outcomes, and while some federal S&T programs 
had of course had great success in achieving the outcomes that the public expected 
from them, the fact is that there existed in the United States at the end of the 20th 
century an extraordinarily modest capacity to develop knowledge, tools, and human 
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resources that would allow the nation to improve its capacity to turn progress in 
S&T into progress toward desired societal outcomes. 

A turning point in achieving high level attention and action came in 2005, when 
President Bush’s science advisor, John Marburger, speaking at the Science and 
Technology Policy Colloquium of the American Association for the Advancement of 
Science, declared that “The nascent field of the social science of science policy needs 
to grow up, and quickly.” His point was that the nation could no longer afford to 
set policy for one of it’s most important areas of public investment on the basis of 
simplistic ideas that had arisen in a very different world, half-a-century ago. The 
National Science Foundation responded to the urgency of Dr. Marburger’s call by 
creating the Science of Science and Innovation Policy (SciSIP) program. 

Committee Question 1. (A) Please provide an overview of the researeh ac- 
tivities of the Consortium for Science, Policy, and Outcomes. (B) 
How are you facilitating interdisciplinary collaborations within 
the Consortium? (C) What new and continuing areas of research in 
the science of science and innovation policy (SciSIP) could signifi- 
cantly improve our ability to design effective programs and better 
target federal research investments? (D) What are the most prom- 
ising research opportunities and what are the biggest research 
gaps? 

Background to CSPO: 

The Consortium for Science, Policy, and Outcomes (CSPO) was conceived in 1997 
during discussions between myself and Michael M. Crow, who was then Executive 
Vice Provost at Columbia University, and formally launched in 1999. The decision 
to create CSPO was made for much the same reasons that SciSIP was created: de- 
spite the overwhelming importance of science and technology in our society, policy 
makers and scholars almost completely lacked the knowledge and tools necessary 
to make informed and effective decisions. CSPO was founded as one small effort to 
begin to reverse this lack of capacity. 

When Michael Crow became President of Arizona State University he asked me 
to move to ASU as well, and gave me the opportunity to help transform CSPO from 
a small research and policy center to a broader consortium with expanded ambition 
and reach. Today this consortium operates at three organizational levels: First, 
there is a core group of fifty or so faculty, researchers, students, and staff who work 
directly in CSPO, mostly in Arizona but with several of us located here in Wash- 
ington, DC. Second, there is a significantly expanded group of collaborators through- 
out ASU as a whole, ranging from many of the university’s top scientists and engi- 
neers, to faculty and students in ASU’s programs on public policy, law, business, 
architecture and design, communications, journalism and even the arts. Third, we 
have deep and persistent collaborations with researchers and students at other uni- 
versities in the U.S. and around the world. Virtually all of our major research 
thrusts are carried out in collaboration with individuals or groups at other univer- 
sities, and CSPO hosts an continual stream of visiting scholars and students, many 
from foreign universities and research institutions, for periods of up to two years. 

In briefly describing CSPO’s major research activities, I want to emphasize a 
point that should be obvious but is often lost in discussions of the Science of Science 
and Innovation Policy. Public support for science and innovation is justified for a 
wide range of reasons, many of which are non-economic. For example, we count on 
science to provide a safe, abundant, and tasty food supply for a growing population; 
ensure the protection of our natural environment and the provision of reliable and 
affordable energy; protect and improve our health; help ensure national security; 
and create new and challenging work opportunities. The reason I belabor this obvi- 
ous point is that in fact we are particularly empty-handed when it comes to under- 
standing how best to design and assess S&T policies aimed at advancing these non- 
economic outcomes. This is the arena where CSPO focuses most of its efforts. 

CSPO is engaged in a wide range of research activities that seek to advance 
knowledge, real-world practice, and human resources in this broad domain of 
science and innovation policy for social outcomes. And I want to gratefully acknowl- 
edge the National Science Foundation’s generosity in providing peer-reviewed grant 
support for many of our most important and I would say high-risk, high-pay-off 
ideas, through a variety of its programs, including SciSIP. 

At the core of all of our research is a commitment to looking at S&T activities 
as part of larger social systems. Trying to understand and assess the outcomes of 
science and innovation by studying and measuring research and development activi- 
ties alone is like analyzing a family’s home life by studying lumber mills and brick 
kilns. What makes a given line of research valuable for society? Of course the 
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science itself must be of high quality, just like a fine home needs to be constructed 
of quality materials. But for investments in science and innovation to support de- 
sired social outcomes, many other elements will come into play: the ways that sci- 
entists choose projects; the culture and organization of research institutions; public- 
private interactions; economic incentives and regulatory structures; public pref- 
erences and behavioral norms — all this and more make up the process by which 
knowledge, innovation, and social benefit are connected. 

1. (A) Please provide an overview of the researeh aetivities of the Consor- 
tium for Seienee, Policy, and Outcomes. 

With this background, let me outline some of our efforts, in four areas of direct 
relevance to the science of science and innovation policy. 

(1) CSPO’s flagship research program is our Center for Nanotechnology in Society 
(CNS), an NSF Nano-scale Science and Engineering Center which has just been re- 
newed for a second and final five-year grant period, under the directorship of CSPO 
co-director Professor David Guston. CNS takes a systems view of technological inno- 
vation to ask: what are the factors that may influence whether an emerging domain 
of technology, in this case nanotechnology, is able to move toward areas of social 
need and desired outcomes? CNS involves multiple universities and researchers 
from multiple disciplines bringing numerous specialties to bear on what we call 
“real-time technology assessment, or a capacity to understand linkages between 
new knowledge, emerging innovations, and societal outcomes — as they are unfold- 
ing. 

Among the many specific research activities encompassed by CNS are relatively 
traditional tools for assessing scientific productivity such as citation and patent 
analysis, as well as proven methods for tracking public opinions and preferences. 
But we also bring social scientists together with nanoscale scientists and engineers 
to reflect on the choices available to them for advancing nanotechnology, and to de- 
velop and discuss future scenarios of nanotechnology-enabled society. We cultivate 
ongoing discussions with the public about potential benefits, problems, and dilem- 
mas of nanotechnology. We bring graduate students working on nanotechnology into 
discussions of science policy and social outcomes. We work with science and tech- 
nology museums to create programs and exhibits that go beyond technical expla- 
nations to help people understand the ways that nanotechnology and society influ- 
ence each other. 

In total, what we are trying to create with CNS is a test-bed for developing a 
more holistic understanding of science, innovation, and social outcomes, where the 
choices made about science, innovation, and their application in society are brought 
out in the open and discussed even at the earliest stages of the innovation process, 
to bring into better alignment the directions of science and innovation, and the aspi- 
rations and needs of society. I also hope it is clear from this brief description that 
standard categories of “basic research,” “applied research,” “education,” and “out- 
reach” are not pursued separately, but are part of an integrated approach at CNS. 

I want to emphasize three elements of U.S. science policy that made this research 
program possible. First was the explicit desire of this Committee and the Congress 
in general, as expressed in the 21st Century Nanotechnology Research and Develop- 
ment Act of 2003, to ensure that nanotechnology advanced along with a capacity to 
understand unfolding social implications. Second was the complementary recogni- 
tion by the National Nanotechnology Initiative, under Mihail Roco’s early leader- 
ship, and the National Science Foundation, that understanding the social aspects 
of nanotechnology should be an important aspect of the overall nanotech research 
agenda. And third was ASU itself, a university that has made huge strides in reduc- 
ing the barriers to true interdisciplinary collaboration, and that is simultaneously 
committed to connecting the work of its faculty and students to the needs of society. 

(2) A second project I want to mention is Science Policy Assessment and Research 
on Climate (SPARC), funded through NSF’s Decision Making Under Uncertainty 
program. SPARC is a collaboration with the Center for Science and Technology Pol- 
icy Research at the University of Colorado, and we are finishing the project up after 
a five-year funding period. SPARC explores a question that lies at the heart of 
science and innovation policy: what makes the results of a scientific research project 
useful, and usable? While the broad context for this project was the nation’s consid- 
erable investment in research related to climate, our research looked at science pol- 
icy decision making aimed at many different problems, including water manage- 
ment, weather and natural hazards, nanotechnology, technological standards, agri- 
culture, and ecology. 

SPARC results reinforce a major point: science policies tend to be more successful 
when they are carried out through institutional arrangements that allow scientists 
and decision makers to understand each other’s needs and capabilities. Fostering 
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close, ongoing, trusting relations between those who produce new knowledge and 
those who might benefit from it seems to be an essential attribute of science policies 
that lead to new knowledge quickly moving into society for public benefit. Drawing 
on the lessons of this major project, we produced a short handbook for science policy 
decision makers, called “Usable Science.” We released this report last April at a 
meeting here in DC that attracted about 100 participants, many from federal agen- 
cies. The handbook is available at: http: ! I cstpr.colorado.edu ! spare ! outreach ! 
spare - handbook / . 

(3) A third project, called Public Value Mapping, or PVM, has been supported by 
the SciSIP program, as well as the V. Kann Rasmussen Foundation and the Rocke- 
feller Foundation. The idea behind PVM draws on my previous point that most pub- 
licly funded S&T activities aim to advance a variety of social outcomes, not just eco- 
nomic ones. PVM finds that these desired social outcomes — what we call “public val- 
ues” — are clearly expressed at many levels across the science and innovation policy 
endeavor — in legislation and laws; in the strategic plans and budget documents of 
R&D agencies; in the websites and press releases of individual R&D programs and 
even projects. 

Because public values are harder to characterize, measure, and assess than eco- 
nomic values, they are often given short shrift both in debates about science and 
innovation policies, and in research to evaluate the outcomes of such policies. Yet 
a key concept for PVM is that the public values associated with science and innova- 
tion policies may conflict with one another, and with economic values. For example, 
a new medical technology may create profit for a corporation and benefit from those 
who have access to the technology, even as it contributes to health care outcome dis- 
parities and over-diagnosis and unnecessary treatment. PVM seeks to unravel and 
clarify such complexities, in order to help view and assess the full range of social 
outcomes tied to science and innovation policies. 

In brief, our research aims first to identify public values across a particular area 
of science and innovation policy. We then analyze how various value statements ac- 
tually relate to each other (for example, are they complementary or contradictory?) 
and assess whether the research activities are in fact organized in ways that may 
allow them to achieve those values. Our work is still preliminary. During three 
years of NSF-supported research, we have completed a set of detailed case studies, 
looking at S&T policy issues such as technology transfer, nanotechnology for cancer 
treatment, and environmental chemistry. One intriguing, but still quite preliminary, 
result of our work is that we think we can say something about the potential for 
a major research program to achieve desired social outcomes based in part on how 
public values are articulated across the program’s various levels and components. 
For example, our study of natural hazards research at the U.S. Geological Survey 
shows a strong coherence among public values expressed by scientists, the agency, 
legislative mandates, and various stakeholders, whereas our analysis of Federal cli- 
mate change research shows considerable diversity and even conflict among values 
within and across these various levels of activity. We are now working to test the 
hypothesis that the relations among public values may in fact be predictive of a pro- 
gram’s performance. If this turns out to hold up after further research, it could offer 
a powerful tool for assessing the capabilities of science and innovation policies. 

(4) As one final example, I want to mention CSPO’s growing work on energy tech- 
nology innovation. This is a cross-cutting theme that works its way into a number 
of our research projects, but I think it helps to communicate our overall approach. 
Consider, for example, solar energy technologies, which may have particular poten- 
tial to serve energy needs in a desert state like Arizona. Yet to understand the po- 
tential for solar energy R&D to contribute to Arizona energy needs, one also needs 
to understand issues of regulatory incentive, land use, water access and availability, 
public lands management, agricultural policies, transmission corridors, military 
bases, Indian reservations, even immigration. Each of these variables may play a 
crucial role in determining the outcomes of solar energy science and innovation poli- 
cies — and policies that do not attend to these variables run the risk of failing to 
achieve their desired social outcomes, regardless of levels of funding or scientific 
productivity. 

1. (B) How are you facilitating interdisciplinary collaborations within the 
Consortium? 

eSPO facilitates interdisciplinary collaboration in three main ways. First, we or- 
ganize our activities around problems, not around disciplines, and then we bring 
into our research teams the expertise that we need to help us understand what’s 
going on and how to make progress. 

Second, as an administrative matter, CSPO is located in ASU’s College of Liberal 
Arts and Sciences, so it does not have a disciplinary affiliation. Our core faculty 
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members have advanced degrees in fields ranging from earth sciences and electrical 
engineering to political science and philosophy. Core faculty are jointly appointed 
between CSPO and a variety of academic units, including the Schools of Life 
Sciences; Government, Politics, and Global Studies; Human Evolution and Social 
Change; Geographical Sciences and Urban Planning; Sustainability; Communica- 
tions; and Social Transformation. If these don’t sound like familiar names for tradi- 
tional academic disciplines, that’s because ASU itself has moved to reorganize 
standard departments into interdisciplinary units in order to bring appropriate in- 
tellectual force to bear on complex problems. 

Third, we have worked hard to cultivate long-term collaborations with natural sci- 
entists and engineers across the university, many of whom are affiliate faculty 
members at CSPO. We work with these colleagues to design new educational and 
research projects and programs that return value both to CSPO and to our science 
and engineering partners. These activities create familiarity and trust that allow us 
to engage in higher-stakes collaborations. For example, many of the major science- 
and-engineering grant proposals submitted by ASU to funding agencies now include 
an integrated set of activities aimed at understanding and enhancing societal out- 
comes. We have even been funded by NSF, partly with the support of the SciSIP 
program, to study the impacts of natural science-social science collaborations in labs 
at ASU and around the world. 

1. (C) What new and continuing areas of research in the science of science 
and innovation policy (SciSIP) could significantly improve our 
ability to design effective programs and better target federal re- 
search investments? (D) What are the most promising research op- 
portunities and what are the biggest research gaps? 

CSPO faculty members have been brainstorming over the past few weeks to de- 
velop a short list of “foundational/transformative” research challenges in response 
to a call for ideas issued by NSF’s directorate for Social, Behavioral, and Economic 
Sciences. Given CSPO’s orientation, our ideas, not surprisingly, are directly relevant 
to the SciSIP program. 

1. Science and innovation policies often aim to help transform existing techno- 
logical systems to achieve particular societal outcomes: for example, to move the na- 
tion’s energy system toward a more economically, environmentally, and geopoliti- 
cally secure technology base; or to move the nation’s health care system to achieve 
better health outcomes at lower cost. New scientific and technological advance are 
obviously going to be key drivers of such transitions. Yet modern societies have very 
little understanding of how to catalyze and steer these sorts of complex system 
changes, and well-intentioned efforts can often lead to unanticipated consequences 
whose benefits are very difficult to assess, as we have seen, for example, in efforts 
to advance alternative biofuels. A key SciSIP research priority should be to 
gain fundamental understanding about the drivers and dynamics of transi- 
tions in complex socio-technical systems. 

2. Science and innovation policies are, in one sense, a bet on the future: that a 
certain type of knowledge or technology will prove useful or valuable. Yet the future 
of social and technological change is impossible to predict in detail. To try to deal 
with this unpredictability, a relatively small number of forward-thinking companies, 
academic units, and non-profit organizations employ a variety of techniques and 
tools that can allow them to better visualize, understand, and discuss a range of 
alternative possible futures. Such activities can inform decision making by helping 
to make clear the broad array and potential implications of scientific, technical, and 
social options and pathways available for addressing social challenges. SciSIP 
should support the study and assessment of existing tools, and the develop- 
ment and testing of a range of new tools, to bring future-visioning tech- 
niques to bear on scienee and innovation poliey making proeesses. 

3. In general, SeiSIP should emphasize support for research and edu- 
cation programs that foster integration between natural scienees and engi- 
neering, and social sciences. Such integration can help to ensure that science 
and engineering activities are conceived and carried out with a realistic under- 
standing of the social context in which knowledge and innovation are pursued and 
applied. In turn, social scientists will gain a deeper, and earlier, understanding of 
the potential futures that cutting edge R&D programs are making possible. The re- 
sult should be a growing capacity to design and conduct science and innovation ac- 
tivities that are better able to contribute to desired social outcomes. 

4. SciSIP should consider supporting the development of a set of case 
studies to identify and characterize the key attributes of S&T institutions 
and programs that strongly link science and innovation activities to de- 
sired social outcomes. Case studies should range across the S&T enterprise, sam- 
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pling a variety of sectors, scales, structures, and desired outcomes. Such a program 
would need to be coordinated to ensure comparability between the methods and or- 
ganization of the cases. Its institutional and programmatic focus would make it dis- 
tinct from, and complementary with, the STAR METRICS approach that NSF and 
sister agencies are already taking. This case-based effort should focus on the devel- 
opment of a set of key organizational principles that science and innovation policy 
makers can use to guide investment strategies and priorities. 

Committee Question 2: Is the Federal Government, speeifieally the National 
Seience Foundation, playing an effective role in fostering SciSIP 
research and the development of a community of practice in 
SciSIP? What recommendations, if any, do you have for the Na- 
tional Science Foundation’s SciSIP program? 

Overall, I believe that NSF is doing a good job in building the SciSIP program 
and community. But this is a very difficult task. The community of researchers 
working in the SciSIP domain is rather small and very diffuse. In fact, it does not 
really identify itself as a single community, but rather as several independent com- 
munities, for example, innovation economics, science and public policy, and science 
and technology studies. So there’s simply not a lot of capacity yet in this domain, 
and what capacity there is needs to be better integrated. Moreover, most of the 
quantitative data available for analysis of science and innovation policy is input-out- 
put data — budget levels, numbers of scientists and graduate students, publication 
numbers, patents, and citations, and so on. Such data can be subjected to highly 
sophisticated data mining and analysis techniques using ever-improved software 
packages designed for this purpose, so it is very attractive to researchers. But this 
kind of input-output data can offer only an incomplete and in many ways distorted 
view of the societal value of the S&T enterprise, a view that does not allow us to 
escape the simplistic beliefs of the past. 

Now it’s clear that those running the SciSIP program understand these problems. 
They brought together a good cross section of the community to help plan the pro- 
gram in the spring and summer of 2005; they have sought to attract grant applica- 
tions from a wide array of researchers; they have organized or otherwise supported 
events to bring together SciSIP researchers to build a sense of integrated commu- 
nity; they have provided grant support to a diverse set of research approaches and 
problems; and they are working through the STAR METRICS program to try to 
build better quantitative data sets that can assist certain types of analjdical work. 
All this is very positive. 

To some extent, however, NSF’s institutional strength is also a weakness here. 
The agency prides itself on its bottom-up approach to setting its research agendas. 
While the SciSIP program does reflect a top-down decision to create a new program 
area, in part as a response to concerns repeatedly expressed by then-Presidential 
science advisor Marburger, the shape and direction of SciSIP has significantly been 
dictated by the existing research community. Much of that community continues to 
work within the input-output model of science and innovation policy, due, as I’ve 
said, to existing data sources and tools. For similar reasons of measurement ease, 
the community also tends to focus on economic outcomes to the significant exclusion 
of the much broader range of societal outcomes that the nation seeks to derive from 
its S&T investment. Because researchers and peer reviewers are drawn from the 
same general communities, such tendencies can be difficult to escape. 

A range of tools are potentially available for building the community and its co- 
herence, and driving the intellectual agenda away from an input-output framework, 
and toward a systems-oriented, outcomes-focused approach. Not all of these tools re- 
quire new money. SciSIP should use program guidelines and requirements to trans- 
form and build the research community; indeed, this year’s program announcement 
is notable for its openness to a wide range of approaches to SciSIP research. SciSIP 
could also consider using some of its budget to support training grants, 
similar in spirit, if not in scale, to NSF’s successful IGERT (Integrated 
Graduate Research and Traineeship) program, as a way to more quickly 
build up capacity. However, if the Committee, and NSF, believe that the science 
and innovation policy research community needs to be significantly larger and more 
coherent, this will probably require more resources. To reinforce my position 
throughout this testimony (also see my comments on “dissemination,” below), any 
claim to a bigger budget must be matched by programmatic design elements to help 
ensure that knowledge created by SciSIP is both usable and used. This would like- 
ly require a commitment to fund integrated Science and Technology Cen- 
ter-type science and innovation policy research organizations that can cre- 
ate and support ongoing interaction between SciSIP researchers and policy 
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makers, perhaps analogous to NOAA’s Regional Integrated Science and Assessment 
program. 

Committee Question 3. Please deseribe the edueation and outreach activi- 
ties of the Consortium for Science, Policy, and Outcomes. 

CSPO sponsors a wide variety of education and outreach activities, ranging from 
formal degree programs and intensive, short-term training activities, to public out- 
reach events and products targeted at science and innovation policy makers. 

1. Graduate Degree Programs 

The ASU Professional Science Masters in Science and Technology Policy was initi- 
ated in 2009. It provides professional education for students seeking advanced pub- 
lic, non-profit, or private sector careers in science and technology policy and related 
fields in the United States or abroad. Students learn essential skills, knowledge, 
and methods for analyzing innovation, expertise, and large-scale technological sys- 
tems. Particular emphasis is placed on the political and societal contexts and im- 
pacts of science and technology policy. The program is a one-year, 30-credit cohort- 
based program designed to attract students of the highest caliber in their early to 
mid-careers. Key learning outcomes of the program include: 

• Understanding of the theoretical foundations of the interactions among 
science, technology, and society. 

• Understanding of US and, where appropriate to a student’s career interests, 
international science and technology policies and the policy processes that 
generate them. 

• Analysis of knowledge systems supporting policy decisions. 

• Analysis of the social and policy dimensions and implications of large-scale 
technological systems. 

• Analysis of scientific and technological innovation systems. 

• Skills in collaborative, team-based analysis of science and technology policy 
problems. 

• Skills in effective professional communication. 

Ph.D. Program in the Human and Social Dimensions (HSD) of Science and Tech- 
nology. Here CSPO collaborates with ASU’s Center for Biology and Society and Cen- 
ter for Law, Science, and Technology to offer a highly interdisciplinary and integra- 
tive program of advanced study. We aim at training scholars and practitioners to 
understand and inform the conceptual and philosophical foundations of scientific re- 
search; to analyze and assess the increasingly powerful roles of science and tech- 
nology as agents of change in society and the economy; and to challenge universities 
to become leaders in fostering the new science and technology policies necessary to 
meet the problems and opportunities of the 21st century. 

The HSD curriculum is flexible, combining a strong, integrated, first-year experi- 
ence, with substantial freedom for students, in conjunction with their advisors, to 
design carefully crafted programs of study relevant to their own areas of interest 
and expertise. The curriculum trains researchers with the necessary skills and prep- 
aration to analyze three key aspects of the study of the human and social dimen- 
sions of science and technology: 1) the historical, philosophical, and conceptual foun- 
dations of science and technology; 2) the social and institutional foundations of sci- 
entific research and technological systems; and 3) the political, ethical, and policy 
foundations of science and technology. 

Research projects of current HSD students supported by CSPO include: 

• Social and ethical challenges of smart grid development 

• Leadership training in graduate science and engineering education 

• Comparative analysis of interdisciplinary research fields in the US and China 

• The emergence and stabilization of legal regimes in online communities 

• The role of non-governmental organizations in energy siting decisions in the 
United States 

• Public values and public engagement in energy policy in the United States 

• The organization and management of international scientific assessment proc- 
esses 

• Connecting knowledge to decision making in water policy 

• information technology in learning & inequality 
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2. Non-Degree Programs and Training 

Ph.D. plus. This integrative, non-degree program offers advanced graduate stu- 
dents in science and engineering the chance to consider how their research relates 
to the world of science policy and the relationship between science, technology and 
societal outcomes. Science and engineering students work with a CSPO faculty 
member to write an additional chapter of their dissertation that explores the social 
implications, political context, or ethical concerns of their work. The Ph.D. plus proc- 
ess is informal, and is arranged by discussions between the student, her or his dis- 
sertation advisor, and the CSPO advisor. Most Ph.D. plus students take one or more 
classes offered by CSPO faculty; attend seminars and other activities sponsored by 
CSPO; and in general interact closely with the CSPO community for an academic 
year or more. 

In the annual DC Summer Disorientation, cohorts of about 15 science and engi- 
neering graduate students spend two weeks in Washington, DC interacting with the 
government officials, lobbyists, staffers, regulators, journalists, academics, museum 
curators, and others who fund, regulate, shape, critique, and study science and tech- 
nology. Students participate in interactive role-playing experiences where they may 
testify at mock Congressional hearings; work under tight deadlines to write briefing 
papers for senior officials; or write op-ed pieces for a demanding editor. The goal 
is to help future scientists develop an understanding of the political and social con- 
text of their research. CSPO has recently expanded this program and now accepts 
graduate students from outside of ASU. 

The Next Generation of Science and Technology Policy Leaders. Here we are seek- 
ing to catalyze a community of early-career science policy scholars who can span the 
terrains of intellectual inquiry and real-world practice, communicate effectively to 
general audiences, and contribute to effective decision making on key issues of 
science, technology, and society. We organized a national competition to select a 
dozen early-career science policy researchers and practitioners (5 years or less since 
Ph.D). This “Next Gen” group prepared draft papers, and each scholar was then 
paired with an early career “communicator” (typically a writer working through new 
media). The scholar and the communicator collaborated to craft a compelling, non- 
scholarly description of the scholar’s work — something that would appeal to a gen- 
eral audience. Next Gen scholars also led a roundtable discussion where each pre- 
sented her/his research to a group of about 40 people at a major CSPO-sponsored 
conference, to allow the scholars to hone the more technical aspects and presen- 
tation of their work, and to interact intensively with an engaged audience. Next Gen 
scholars are now working on two versions of their research papers, one for a policy- 
making audience, and one for an academic audience. This project was supported by 
grants from NSF’s programs on Science, Technology, and Society, and Informal 
Science Education. 

3. Outreach 

CSPO views outreach as an inte^al part of its operations at all levels — not as 
a separate, add-on, or late-stage activity. As described above, our research and edu- 
cation programs often involve policy makers, members of the public, and scientists 
and engineers, and so also serve an outreach function by creating and strengthening 
links and communication between CSPO scholars and these other groups. Indeed, 
in many cases it is difficult to know where research ends and outreach begins. For 
example, much of our work on energy innovation policy is presented to policy mak- 
ers and the media in briefings and policy reports at the same time as it is written 
up for academic audiences (see: http:! I www.cspo.org I projects I eisbu I). Similarly, 
SPARC involved numerous workshops that brought scientists and science policy 
makers together in a way that enhanced both communication and learning. 

The integration of outreach and education is apparent in CSPO’s growing collabo- 
ration with science museums and science centers. We view these collaborations as 
ways of reaching wider audiences and increasing the ability of our graduate stu- 
dents — social scientists as well as natural scientists and engineers — to communicate 
to broader audiences. Our Center for Nanotechnology in Society has fostered a na- 
tional strategic partnership with the NSF-funded Nano-Scale Informal Science Edu- 
cation Network to develop programs and exhibition materials and plans that incor- 
porate societal interests and outcomes in communicating about emerging tech- 
nologies. CSPO opens science communication opportunities for scientists and engi- 
neers through its monthly Science Cafe series with the Arizona Science Center and 
incorporates museum-floor experience into its integrated training of doctoral sci- 
entists and engineers. CSPO is also working with the Museum of Science, Boston 
and the National Academy of Engineering to plan a national educational campaign 
to focus on climate change and engineered systems, to prepare the next generation 
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of engineers, citizens, and leaders to meet the challenge of adapting the nation’s 
technological infrastructure to climate change. 

Overall, we are continually engaged in a wide variety of efforts to make our ideas 
accessible to the public and policy makers, through informal and formal meetings 
and briefings in the Phoenix area and in Washington, DC; through “handbooks” for 
decision makers; through ongoing contact with the media; as well as by writing op- 
eds and articles for non-technical magazines, websites, and blogs. We have just re- 
ceived a small supplement to our SciSIP grant on Public Value Mapping to produce 
engaging, instructional web-based videos for science policy practitioners. New out- 
reach products and activities are promoted via CSPO’s monthly electronic news- 
letter, which goes to over 3000 people in academia, government, and industry. In 
all, I think it is fair to say that CSPO views outreach, education, and research as 
equally necessary foundations for pursuing its mission. 

Committee Question 4. How ean the dissemination of SeiSIP researeh find- 
ings be improved so that policymakers are better informed of the 
current state of research? Are there best practices that can be im- 
plemented by the Federal government and/or the research commu- 
nity to improve the incorporation of science and technology policy 
research into the decision making process? 

SciSIP program officers, in collaboration with their grantees, with organizations 
like the American Association for the Advancement of Science, and with other fed- 
eral agencies, has made an impressive effort to ensure that research results are 
made available to science and innovation policy makers, through the SciSIP website 
and listserve, and through a variety of workshops, including one to be held this com- 
ing December. 

SciSIP and NSF more broadly face something of a dilemma here, however. As I’m 
sure the Committee well appreciates, academic researchers are generally not re- 
warded for communicating their work to policy makers, or even for making the re- 
sults of their work comprehensible to non-experts. I’m extraordinarily fortunate to 
work at a university where this is not the case. Moreover, given the fundamental 
nature of much of the research supported by SciSIP, the extent to which project re- 
sults can translate into results immediately useful to decision makers may be highly 
variable. At the same time, it’s fair to say that science and innovation policy deci- 
sion makers may not always be either receptive to, or able to act on, the results 
of research conducted under SciSIP. 

In line with many of the comments I’ve already made, and consistent with re- 
search done by CSPO and many other groups, the best way to further improve the 
value of SciSIP research for decision makers would be to increase the level of inter- 
action between the researchers and decision makers. This point should not be inter- 
preted as a criticism of the current SciSIP program, which as far as I can tell is 
effectively pushing the boundaries of typical NSF practice, and working at the limits 
of its human and fiscal resources, to try to maximize dissemination. 

Yet ensuring that researchers are providing knowledge that decision makers can 
actually use is not only a matter of “dissemination,” it is also structural. For SciSIP 
results to be both usable and used, researchers and decision makers must each come 
to understand the needs, capabilities, and languages of the other — a process that 
we have termed, in our SPARC project, “reconciling the supply and demand of re- 
search.” Such a reconciliation takes time and ongoing interaction. It can certainly 
be pursued along multiple paths — through joint committees, workshops, personnel 
exchanges, interviews and surveys, and so on — but the key is ongoing and meaning- 
ful interaction leading to mutual understanding. An NSF research program, even 
one advanced with the creativity and vigor that characterizes SciSIP, is unlikely to 
be able, by itself, to provide the sort of institutional infrastructure that leads to the 
production of consistently usable knowledge. The idea of integrated SciSIP centers, 
previously mentioned, could be one way to create a greater capacity to move ideas 
into use. Federal agencies and programs that sponsor mission-oriented research, 
and that have a proven record of producing usable knowledge, might also be able 
to play a role here to help achieve the necessary integration. 
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Committee Question 5: What are the fundamental skills and eontent knowl- 
edge needed hy SeiSIP researehers and praetitioners? What are 
the backgrounds of students pursuing graduate degrees in science 
and technology policy, and what career paths are sought hy these 
graduates? Is the National Science Foundation playing an effective 
role in fostering the development of science and technology policy 
degree programs at U.S. universities? If not, what recommenda- 
tions, if any, do you have for NSF and/or the universities with such 
programs? 

As I’ve suggested, the domain of SciSIP research and practice cannot and should 
not be defined by any particular set of skills or area of knowledge. In fact, given 
the complexity and diversity of the challenges facing SciSIP policy makers, it will 
be important to keep the field as open and flexible as possible, where the necessary 
skills and knowledge are determined based on the problem at hand, and on the evo- 
lution of the field itself, rather than some arbitrary boundary. I’ve already men- 
tioned the varied backgrounds of CSPO’s core faculty group, and our graduate stu- 
dents are if anything even more diverse, coming to us with degrees in business, in- 
formation systems, science and technology studies, astrophysics, political science, 
law, English, public policy, library and information science, philosophy, physics, bi- 
ology, environmental science, geology, anthropology, sociology, and industrial man- 
agement. 

Graduate training in science and technology policy is also diverse, occurring in 
many different types of programs, with many institutional and administrative ar- 
rangements, in many U.S. universities. There is no standard-model science and 
technology policy graduate degree, and given the complexity of the field perhaps 
that is just as well, but it does create challenges in terms of attracting resources, 
creating an identity, and setting priorities. Similarly, while many career paths are 
open to those who have advanced training and degrees in science and technology 
policy, there is no formula for how to build or advance a career in this field, as there 
is in, say, law, medicine, or engineering. In CSPO’s brief experience with graduate 
education, we do see our students and post-docs progressing on traditional academic 
paths, but they are also going into the private sector, working at nongovernmental 
organizations, and taking up positions in government agencies and think tanks. I 
also want to emphasize the importance that we place on “continuing education” via 
our professional Masters program, which we hope will reach mid-career profes- 
sionals already working in areas related to science and technology policy, and equip 
them with tools to do their jobs more effectively, or to move into more complex jobs, 
in the public, private, and nongovernmental sectors. 

As I discussed in my response to Question 2, NSF’s SciSIP program, as well as 
its Science, Technology, and Society Program, are working hard to build a sense of 
community and identity among science and technology policy researchers, and to 
provide support for research across a broad domain of problems and applications. 
However, as discussed at length by about 75 members of the community at this 
summer’s Gordon Conference on Science and Technology Policy, the traditional aca- 
demic structure of universities remains a considerable obstacle to building long-term 
capacity in the field, and most science and technology policy programs exist in the 
margins and spaces of standard disciplinary schools and departments. I am fortu- 
nate enough to work at a university whose leadership has a strong commitment to 
cultivating interdisciplinary, problem-based research that can link knowledge cre- 
ation to solutions for complex societal problems. Yet even at ASU the long-term fu- 
ture of science and technology policy research probably depends on finding a way 
to more closely knit CSPO into the fabric of the formal academic units on campus. 

One conclusion here is that NSF’s ability to foster the development of the field 
of science and technology policy is partly dependent on incentivizing universities to 
recognize SciSIP as a field worth cultivating. While the SciSIP program is certainly 
of a scale sufficient to mobilize and motivate individual researchers working on 
science and innovation policy, it is probably not big enough to get the attention of 
university administrators. I have already emphasized the potential value of apply- 
ing an integrated Science and Technology Center model to building the SciSIP com- 
munity and moving its research results into use. An NSF commitment to supporting 
one or more such centers would also send a strong signal to university leaders that 
the science of science and innovation policy is a national priority, deserving of 
strong focused effort and investment from our universities. 
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Biography for Daniel Sarewitz 

Daniel Sarewitz is Professor of Science and Society, and co-director and co-founder 
of the Consortium for Science, Policy, and Outcomes (CSPO), at Arizona State Uni- 
versity {http:! I www.cspo.org). His work focuses on revealing the connections be- 
tween science policy decisions, scientific research and social outcomes. How does the 
distribution of the social benefits of science relate to the way that we organize sci- 
entific inquiry? What accounts for the highly uneven advance of know-how related 
to solving human problems? How do the interactions between scientific uncertainty 
and human values influence decision making? How does technological innovation in- 
fluence politics? And how can improved insight into such questions contribute to 
real-world practice? He is the author of Frontiers of Illusion: Science, Technology, 
and the Politics of Progress (Temple, 1996), an exploration of the public m 3 dhs that 
underlie decisions about science and technology; the co-editor of three other books; 
and the author of many articles about the interactions of science, technology, and 
society. In addition to scholarly journals his work has appeared in The Atlantic 
Monthly, The New Republic, and many newspapers; from December 2009 until Sep- 
tember 2010 he wrote a monthly column on science policy for the journal Nature. 
His work has also received featured coverage on NPR’s Morning Edition, in the New 
York Times and the Chronicle of Higher Education. From 1989-1993 he worked on 
R&D policy issues for the U.S. House of Representatives, first as a AAAS Fellow 
in the office of Congressman George E. Brown, Jr., and then as a staffer on the 
Committee on Science, Space, and Technology. He received a Ph.D. in Geological 
Sciences from Cornell University in 1986. He now directs the Washington, DC, office 
of CSPO, and focuses his efforts on a range of activities to increase CSPO’s impact 
on federal science and technology policy processes. His new book. The Techno- 
Human Condition (co-authored with Braden Allenby; MIT Press) will be published 
in March 2011. 

Chairman LiPiNSKl. Thank you, Dr. Sarewitz. And it is the be- 
ginning of votes, but we should be able to get to the testimony in 
here. Dr. Murray. 

STATEMENT OF FIONA MURRAY, ASSOCIATE PROFESSOR OF 

MANAGEMENT, TECHNOLOGICAL INNOVATION & ENTRE- 
PRENEUR GROUP, MIT SLOAN SCHOOL OF MANAGEMENT 

Dr. Murray. Okay. Thank you very much. Thank you. Chairman 
Lipinski and other Members of the Subcommittee, for the oppor- 
tunity for being here. My name is Fiona Murray as you heard be- 
fore I am a Professor of Innovation and Entrepreneurship at the 
MIT Sloan School and I am also Associate Director of the MIT En- 
trepreneurship Center. 

Now to start my remarks I thought I would just describe the per- 
spective I bring. Briefly, I am the grateful recipient of two SciSIP 
Grants. I worked on what I would think of as a SciSIP oriented re- 
search agenda for more than a decade, although I really only dis- 
covered the SciSIP research community in about 2006. As a faculty 
member of a business school I also engage on these issues with 
managers, scientists themselves who are also interested, sometimes 
at the lower levels, in how to organize effectively to ensure the pro- 
ductivity and impact of their research. 

I should also just say something about my own training. I have 
a background, a Bachelor’s, Master’s, and Ph.D. in Chemistry. That 
is a very unusual training for somebody who does SciSIP. I think 
it enables me to bring a unique understanding of the bench science 
to this research, although as I do note in my written remarks I am 
not sure that this is an ideal past to learn the rigorous social 
science methods that one really needs. I have had to rely on again 
self-education and some very patient co-workers to get me over 
what I think is a quite high bar to make a serious contribution to 
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this endeavor, and in particular to do it in a way that contributes 
to the policy and the scholarly debate. 

I want to just take my time to see if I can make three points. 
I will make two if time — if only time permits. Something about the 
vision of SciSIP that — from what I think that means about the 
kinds of gaps there are in the research. And then also how I think 
the scientific community might more effectively be organized to 
really have an impact in terms of research links to the community 
and in particular education. 

So I think that SciSIP is not completely about doing science and 
technology analysis. I think there’s already an excellent scholarship 
describing policy initiatives, the government attitude toward 
science, and the politics of science and innovation policy. But I 
think that what SciSIP brings, which is unique, is this sort of sci- 
entific lens to the problem. And what I mean by that is that it is 
a serious and I think important attempt to undertake causal anal- 
ysis and evidence-based analysis aslang whether and how par- 
ticular policy interventions actually have an impact, whether it is 
in the short run or the long run. And so I think that good scientific 
research defines impact richly: it is about the level of the rate and 
the direction of scientific progress and innovation, but it is also 
about long run impact on economic growth. 

But I also want to emphasize this causal piece of what kinds of 
policies we think make a difference. I think that at its best, SciSIP 
defines policy broadly but precisely in particular research in- 
stances. And so it can mean everything from high level national 
policies, the laws, but also agency implementation processes, agen- 
cy selection processes, but below that, community behaviors, things 
like the Bermuda Laws and so on. And even at some microlevel, 
lab level choices around how we choose to organize the scientific re- 
search at the ground level. 

I think a key approach to SciSIP has been grounded in two re- 
cent developments. One is the data development which has already 
been discussed. But I would also say it has been enabled, actually, 
by a massive scientific data infrastructure investment, so some of 
my own work has really been enabled by investments in things like 
GEN Bank and that ability to interrogate genetic data to then do 
science policy analysis. 

But also I think a second piece of social science methods is in 
program evaluation. Actually, you are familiar with this from the 
work that you do on evaluating education policy. But I think that 
the ability to use experiments and causal analysis and so on from 
that policy evaluation tool kit is extremely important to pushing 
SciSIP forward. 

So I think that SciSIP has really been critical and attracted seri- 
ous cause, but in my view there are still some gaps. To pick up the 
“straw and bricks” analogy, it strikes me that while we need bricks, 
if we want to cross the bridge from data to understanding we actu- 
ally have to build a bridge with those bricks. And what does that 
mean? I think that does mean more analysis as well as just meas- 
urement. I think at the moment a lot of the scientific work, includ- 
ing my own, is intrinsically focused on biologists and on funding at 
the National Institutes of Health. That is critical, but not the only 
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arena, and I think that there is — we do need to understand how 
other disciplines and other agencies are working. 

I think that there has been a focus on national-level rules and 
specific agencies and less on these community-level choices about 
how to organize structure, collaborations, and more informal ef- 
forts. I think we also need to focus on distributional issues. So not 
just how many more papers are produced, but what kind of a 
breakthrough or everyday science, what kinds of research, are they 
American or foreign, and so on. I don’t think we have focused 
enough on that. 

And let me in the last few seconds just say something about the 
SciSIP community. I think that the community actually needs to 
do more of its own bottom-up organizing. The NSF has done a tre- 
mendous job in kind of structuring it in a top-down way, but that 
is a huge amount of work for one agency to do. And I think as a 
community we need to do a more bottom-up in order to both engage 
in more knowledge exchange among ourselves and to focus on edu- 
cation. And I think the educational imperative at the Ph.D. level 
does need to be organized across a number of campuses. And then, 
as I think at the policy level of our links, the policy makers again 
have to be organized in a more community-based way. So I would 
suggest that that needs to be done through a consortium of univer- 
sities but with this tripartite mission of research, education, and 
then links to policy. And I will leave my remarks there. Thank you 
very much. 

[The prepared statement of Dr. Murray follows:] 

Prepared Statement of Fiona Murray 

According to the National Science Foundation (NSF), the Science of Science & In- 
novation Policy (SciSIP) program “supports research designed to advance the sci- 
entific basis of science and innovation policy i. The program is an important and 
bold attempt to build a strong intellectual foundation for science and technology pol- 
icy making regarding the laws and rules that shape the institutional environment 
in which scientific research and innovation takes place. It does so by adopting re- 
cently developed, leading-edge methodological approaches based on both large scale 
empirical data analyses and complementary qualitative analyses. The explicit goals 
of the program are to fund research that “develops, improves and expands models, 
analytical tools, data and metrics that can be applied in the science policy decision 
making process”. From my perspective as a SciSIP scholar, I conceptualize the 
SciSIP agenda as the systematic, evidence-based and causal analysis of the impact 
of policy interventions on the rate, direction and impact of scientific knowledge pro- 
duction and innovation. If successful in research and in coupling to policy decisions 
then this agenda will enable Federal and state policymakers, as well as others en- 
gaged in shaping the production and translation of scientific knowledge (including 
scientists themselves, universities. Foundations and scientific communities), to de- 
sign more effective policies and practices that ensure that investments in science 
and innovation have rapid and extensive scientific, social, and economic impact. 

In this testimony I lay out my personal views of the SciSIP program from the per- 
spective of an NSF-SciSIP scholar (and grant recipient), and as a Faculty member 
in a leading School of Management who engages routinely with scientists concerned 
with the impact of their research, policy students as well as MBA students and ex- 
ecutives hoping to work effectively at the academic-commercial interface. 

In what follows I examine some recent breakthroughs that have enabled SciSIP 
research, outline some of the key research emerging from SciSIP to date and critical 
gaps. I then turn my attention to what I observe as the need for greater community 
building and finally, the potential for a significant educational initiative. 


^Accessed from http: ! I www.tisf.gov ! funding Ipgm^summ.jsp^pims -id=501084&org=she 9/16/ 
2010 . 
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The notion that there can be a “science” of science and innovation policy is rel- 
atively recent (Marburger 2005). There is a long and distinguished traditional of 
science policy research nonetheless, the current focus on measuring the causal influ- 
ence of science and innovation policy levers at different levels (national policy, agen- 
cy interventions as well as community and lab-level actions) can be linked to ad- 
vances in economics and related fields in the early 1990s. During this period, lead- 
ing economic historians including Paul David, Joel Mokyr and Nathan Rosenberg 
developed critical conceptual breakthroughs in understanding the economics of 
science and innovation as grounded both in institutions (policy levers) but also in 
the micro-level behaviors and incentives of scientists and engineers themselves. 
Building on economics as well as the sociology of science, they came to view Science 
as a distinctive institution in several ways: as a knowledge production system, as 
an input into technological innovation, and as a reward system. 

The empirical promise of this conceptual agenda was taken forward by a group 
of economists and sociologists who aimed to evaluate the impact of public policies 
on research behavior, research outputs, and associated economic outcomes 
(Marburger, 2005; Jaffe, 2006). In following this agenda, scholars confront a number 
of key challenges. In particular is it possible to separate the influence of a particular 
policy or institution from the underlying nature of the scientific knowledge that is 
being developed? To put it more simply, in the policy “whodunit” it is often hard 
to say whether it is the policy that had the effect of speeding up scientific progress 
in a particular area or a chance in our understanding of a scientific problem. With- 
out a parallel universe for policy experiments, when one observes the production or 
diffusion of a piece of knowledge within a given policy environment, one cannot di- 
rectly observe the counterfactual production and diffusion of that knowledge had it 
been produced and diffused under alternative policy conditions. To resolve these 
challenges, SciSIP scholars have combined methodological advances in program 
evaluation- particularly a “natural experiments” approach — with novel data tech- 
niques. The experiments-based approach (with which the committee is likely famil- 
iar from its work on education) relies upon methods pioneered in public finance and 
program evaluation (Meyer, 1995; Bertrand, Duflo, and Mullainathan, 2004; Angrist 
and Pischke, 2008). To complement these methods, SciSIP scholars have made ex- 
tensive use of novel datasets including data on publications, patents and most re- 
cently citations. This approach uses these “documents” as the core objects of anal- 
ysis, assuming that they represent “pieces of scientific knowledge,” and citation 
analysis to investigate the impact of institutions on the cumulativeness of discovery 
and innovation (Garfield, 1955, De Solla Price, 1970; Jaffe, et al, 1993; Griliches, 
1990, 1998). When placed within a framework to evaluate science and innovation 
policy, these elements constitute a robust approach to analyzing and tracking the 
causal impact of public policies on science and innovation inputs and outputs. 

The power of the emerging SciSIP agenda is to incorporate these novel approaches 
and therefore move beyond description and observation of science at work or par- 
ticular policies towards the more systematic analysis of particular institutional 
interventions. Thus pioneering SciSIP research typically combines three elements: 

i) Providing clear theoretical foundations for understanding the ways in 
which institutional change (at any level) might influence the behavior of 
scientists and therefore the rate and direction of their knowledge produc- 
tion. 

ii) Building careful empirical designs that enable causal analysis, and under- 
taking these empirical studies using systematic data gathering methods at 
different levels (including quantitative data but also including qualitative 
studies). 

iii) Grounding the analysis in a deep understanding of the phenomenon — the 
details of the particular policy changes or organizational choices as well as 
the ways in which these shape scientists daily life. 

As a contributor to the broader SciSIP agenda and approach, my research in the 
past few years has focused on the conflicts and compromises shaping the boundary 
between academic science and the commercial world — especially the impact of intel- 
lectual property (IP) rights and IP licensing strategies over basic scientific research 
in areas as diverse at human genetics, stem cells and cancer biology. More recently 
I have expanded my research to examine the community and organizational-level 
interventions that scientists can make including understanding how research qual- 
ity is governed (through retractions) and how projects are selected and evaluated). 
In my own work, I have found that my training as a scientist provides aids in the 
third element of the SciSIP approach but my work is strongly based on the theories 
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and methods of economics and sociology of science and therefore links the three as- 
pects outlined above. 

A research project of mine recently completed with a series of co-authors illus- 
trates the SciSIP approach to the analysis of science policy. It was designed to adju- 
dicate one policy element of the institutional complex — the impact of intellectual 
property rights on research tools (and the licenses that shape access to such tools) 
on scientific productivity and diversity. Rather than theorizing broadly, it focuses 
specifically on one controversial episode in the genetics community initiated by the 
discovery, patenting and then exclusive licensing of mouse genetics technology (the 
Oncomouse approach and the related Cre-lox approach) and the subsequent licens- 
ing agreement made among DuPont, the Jackson Laboratories and the National In- 
stitutes of Health to enable greater access to these key research tools. 

In The Oncomouse that Roared (Murray 2010), I take a qualitative approach to 
the question of whether and how the Oncomouse patent influenced the scientific 
community. Rather than compare the mouse genetics community to another sci- 
entific field (which may have any number of inherent differences), I compare the pe- 
riods before and after the Oncomouse patent was granted and licensed. For 3-4 
years, with no intellectual property rights yet granted the mice were subject only 
to the informal norms that characterize a competitive, but collegial, scientific com- 
munity. After the grant of the patent, DuPont (exclusive licensee) strongly enforced 
its property rights on scientists. Through detailed interviews and documentary anal- 
ysis comparing the pre- and post- patent period, I follow the SciSIP approach and 
closely analyzed the impact of the Oncomouse patent on mouse geneticists. I find 
that some scientists reluctantly acquiesced, dealing with complex contracts. Others 
defied DuPont, sharing mice informally in the face of opposition from their univer- 
sities. Behind the scenes other more complex changes were also taking place as sci- 
entists sought to reshape the role patents in their scientific life. This is reflective 
of broader changes in the scientific community in the face of higher levels of com- 
mercial interest and engagement and the resistance to the encroachment of high- 
powered commercial interests. Such a grounded perspective highlights the impor- 
tance of understanding how scientists respond to policy interventions and has a 
number of policy implications. However it also raises a more SciSIP-oriented ques- 
tion about the causal impact of the compromise (when the NIH persuaded DuPont 
to sign a Memorandum of Understanding making Oncomice open for experimen- 
tation) on the level and type of research using these genetically modified mice i.e. 
do such policy interventions shape the rate and direction of science. 

I examine the causal impact of these shifts to greater openness in “Of Mice and 
Academics” (Murray et al. 2010). The “dependent variable” in this paper is the level 
and type of scientific research publications that use genetically engineered mice in 
each year from 1990 until 2006 — based on a dataset of over 20,000 publications that 
are coded by their level of basicness, the rating of the journal in which they are pub- 
lished, the rank of the school affiliations of the authors etc. The “independent vari- 
able” is the timing of the policy shift in the openness of particular types of 
transgenic mice (Onco mice and Cre-lox mice). To aid in the interpretation of the 
data we also include a control group of papers that build on mice never influenced 
by intellectual property rules. A central idea of this research design is that while 
research discoveries (such as engineered mice) are developed at a given point in 
time, their use by subsequent researchers takes place over time. This insight moti- 
vates a differences-in-differences approach to the analysis of follow-on scientific re- 
search: If the policy environment governing the incentives and/or ability to build on 
published discoveries changes over time (and affects only some discoveries but not 
others), it is possible to identify the impact of the policy change by examining how 
the pattern of follow-on research (captured in published articles) changes after the 
policy intervention. In other words, policy changes that impact one group of articles 
and not another can constitute a natural experiment. This paper exemplifies the 
SciSIP approach by linking (microeconomic) theory about the way researchers re- 
spond to openness, with data/empirics that allow for causal analysis, and a suffi- 
ciently detailed understanding of the policies and practices of scientists to enable 
appropriate research design. We find that the NIH MoU did indeed increase not 
only the level of research using these mice but also spurred a greater diversity of 
researchers to move into the field, follow novel paths and take new approaches. 

Taken together these two papers address questions of how institutional and orga- 
nizational changes shape the rate and direction of scientific knowledge. They follow 
the three key elements of the SciSIP approach by carefully and precisely focusing 
on the phenomenon at hand, using that detailed understanding to link theories of 
scientists’ behavior to careful data, and building empirical strategies in a way that 
enables causal analysis, normative conclusions and theoretical contributions. 
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ASSESSING THE GAPS IN SciSIP KNOWLEDGE 

As outlined above, the SciSIP agenda presents far reaching research opportunities 
for scholars whose goal is to contribute to the social sciences, to our understanding 
of science and innovation in the economy and to have policy impact. A number of 
significant gaps in the current state of knowledge remain and can be usefully con- 
sidered around the organizing framework laid out below. This describes SciSIP re- 
search according to the level of analysis at which the policy interventions are taking 
place: national rules and regulations, agency-level interventions, community norms 
and practices and organizational actions. I then propose three cross-cutting ques- 
tions that apply to each level (see below). To illustrate this perspective and the gaps 
it reveals, I first describe research on high level rules and regulations then move 
to more micro-level analysis of organizational interventions. 

• National rules and regulations: This includes research on the effectiveness 
of national rules and regulations on the rate and direction of scientific 
progress. A major area of focus includes research on the influence of the 
Bayh-Dole Act on university researchers (Owen-Smith and Powell 2003). In 
my own recent research, we have examined the impact of US regulations with 
regards to the funding of research in the area of human embryonic stem cells 
(Furman, Murray and Stern 2010). Gaps at this level of analysis remain with 
regards to the role of international rules and regulations on science in the 
United States, and the ability of U.S. researchers to remain highly competitive 
and at the knowledge frontier in the light of growing global spending on sci- 
entific research. In addition it would be valuable to understand how the par- 
ticular funding levels, structure and incentives of university systems in dif- 
ferent countries impact downstream outcomes, such as scientific production, 
firm founding, and health & welfare, and how they contour the impact of gov- 
ernment policies such as those related to intellectual property rights. 

• Agency or University-level rules and norms: Funding agencies, especially 
the Federal government, have a variety of opportunities to shape the rate and 
direction of scientific progress. Both areas have up to now been poorly under- 
stood. Recent work funded by SciSIP has made significant progress along 
these two dimensions but gaps remain. In particular the influence of non-Fed- 
eral funding sources particularly corporate funding and the growing founda- 
tion funding is poorly documented and understood. 

o Shaping the Direction of Research: Funding agencies, as they select 
among research projects and shape the expectations and controls they 
place on researchers have a variety of opportunities to influence knowl- 
edge production. This has often been thought of as a black-box with the 
scientific community utilizing the peer review system as the best mecha- 
nism to self-regulate and shape direction. Pioneering analysis by my MIT 
colleagues shows that exceptional scientists are much more likely to 
produce innovative breakthrough science when using long-term grants 
that allow them exceptional freedom in the lab (Azoulay et al, 2010) 2 . 
This study raises the question of how researchers are encouraged to move 
into new and emerging research areas, and how to encourage ideas at the 
high-quality high-risk tail of the distribution. 

We must encourage more research to understand the impact of funding 
choices and funding incentives on the type of research outcomes. This 
agenda could also benefit from the analysis of scientists outside the U.S. 
in settings where different types of incentive systems exist. In line with re- 
cent interest in Challenges (prizes) as an alternative incentive mechanism, 
we should also extend this analysis to include other funding mechanisms 
or reward systems. 

o Shaping the Disclosure and Sharing of Knowledge and Materials: 

Funding agencies have an opportunity to shape the rate and effectiveness 
with which knowledge that is generated as a result of grant-making is 
shared among scientists and is diffused into the economy along produc- 
tive routes. Among the most important and controversial rules shaping 
such impact of scientific research are the rules around intellectual prop- 
erty rights. This has been the topic of vigorous debate particularly with 
regards to the increasing levels of patenting within the scientific commu- 
nity. This is the research arena in which SciSIP researchers have made 


^They do this by comparing the research profiles of similar biologists some of whom receive 
more open ended long-term funding from Howard Hughes while others receive more traditional 
ROl-style grants from the NIH. 
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one of the greatest contributions, with their research informing policy 
discussions at the National Academies of Science, within the National In- 
stitutes of Health (NIH) and elsewhere. In particular, research has ex- 
plored the impact of patenting on the rate at which that research is dif- 
fused within the scientific community and on the rate at which commer- 
cial or socially-beneficial products are developed (Murray and Stern 2007; 
Huang and Murray 2009; Walsh et al. 2003, 2005). Extensive research 
documents the impact of IP, licensing and material sharing practices on 
scientists, but gaps in our knowledge exist with regards to the impact of 
these policies on both scientific knowledge production and economic im- 
pact (few studies examine both with Williams (2010) a notable exception). 
We also have a less systematic understanding of how to design the “intel- 
lectual commons” in an efficient and effective manner so as to promote 
and rapid follow-on research and commercialization. There is also a sig- 
nificant opportunity to extend these studies beyond the study of life sci- 
entists to explore differences across research communities in a range of 
disciplines such as chemistry, computer science, materials science etc. 

• Community level activities: The policies and practices that emerge from 
the scientific community also play a critical role in scientific progress and im- 
pact. Thanks to more systematic analysis of resource-sharing arrangements 
both informally (see Hauessler et al. 2009; Waltsh et ah, 2005) and through 
formal mechanisms such as Biological Resource Centers, there is definitive 
evidence that investments in community-based infrastructure such as mate- 
rials repositories and data repositories have a significant positive impact on 
the rate of scientific progress by enabling access, certification and sharing 
(Furman and Stern 2010). More recent analysis of the self-governance of sci- 
entific communities through the system of retractions has also pointed out the 
role of the community as a crucial analytic lens (Furman and Murray 2009). 
In another stream of research grounded in organizational theory and soci- 
ology, scholars have examined whether and how different community struc- 
tures emerge in order to undertake the complex task of horizontal collabora- 
tion (e.g. Powell et al. 2004, O’Mahony and Bechky 2008) and collective work 
(Ferraro and O’Mahony forthcoming). 

At this level of analysis, critical questions remain unanswered: how are sci- 
entific communities formed? How do they coalesce around new research areas 
and what role might policy-makers play in such community formation? For ex- 
ample do mechanisms such as those used in DARPA enable community build- 
ing and how does this shape the long run effectiveness of scientific commu- 
nities? 

• Organizational Interventions: Scientific research is an activity increas- 
ingly undertaken by collections of scientists organized into teams, networks, 
collaborations and networks. Recent scholarship highlighted the potential for 
significant productivity benefits of specific organizational choices (Cummings 
and Keisler 2005, 2007, Wutchy et al. 2007, Jones et al. 2008)®. Recent work 
on open source computer science communities highlights the complex and so- 
phisticated nature of the organizational and governance choices that these 
groups of scientists can make (Dahlander and O’Mahony forthcoming) and 
their implications for the nature of the knowledge production (MacCormack 
et al. 2006, 2008). However, there remained only limited research that exam- 
ines the organizational choices of scientists for specific research projects — they 
choice of collaborations, organization of tasks in the lab, governance of the lab- 
oratory. In part this gap arises because of the historic perspective of the sci- 
entist as “loan genius." Moreover, the strong sense of autonomy among the sci- 
entific community has limited the research on choices that scientists them- 
selves make. 

Opportunities for further research also cut across these levels of analysis with 
three of key importance: 

i) On what field has the SeiSIP researeh been foeused? In other words, 
is the analysis focused on a particular scientific discipline or sub-field e.g. 
biology, high-energy physics, nanotechnology? In my opinion, too large a 
share of current SciSIP research (including my own) highlights the biolo- 
gists to the exclusion of other arenas. For example we have little knowl- 
edge of the influence of policies on material scientists who, like biologists. 


®Ben Jones, a leading scholars in the SciSIP field and author of several key papers in this 
area is currently a senior economist at the Council on Economic Advisors. 
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rely on complex materials, data, images etc. Our knowledge of chemistry, 
computer science & engineering remains fragmented. 

ii) On what outcomes has the SciSIP research been focused? Is the 
analysis focused on academic publications, patents or marketed products? 
As noted above, these outcomes are now well documented in the SciSIP lit- 
erature. More emphasis however should be placed on linking up different 
measures i.e. publications and patents and finding metrics that capture 
commercializable or commercialized products (see Williams 2010) or meas- 
ures that capture the broader knowledge landscape such as recent analysis 
of the patenting of the entire human genome (Jensen and Murray 2005). 
In this regard, data on licensing would be more valuable than patenting 
data alone and yet such information (for ideas developed using Federal 
funding) is not available. I would strongly recommend that this be changed 
to facilitate greater and more systematic analysis using measures closer to 
the outcomes and impacts of economic interest. 

iii) On what part of the outcome distribution are SciSIP analyses fo- 
cused? It is important that SciSIP researcher evaluate which researchers 
and which institutions were most affected by particular policy interventions 
rather than simply highlighting the average impact of particular policies. 
How do policy interventions impact the distribution of knowledge outcomes? 
While there may be no impact on the mean perhaps interventions influence 
the distribution of outcomes — with more high and low quality research. How 
might policy-levers all levels influence different researchers? What is their 
marginal impact on different groups of scientists: those at elite highly fund- 
ed schools versus elsewhere, or those with international co-authorship ties 
Studies that emphasize these distributional outcomes should be encouraged 
by SciSIP because it is from the richness and diversity of the scientific com- 
munity that novel breakthrough outcomes arise. Studies could also fruitfully 
include analysis of the differential impact of policies on male versus female 
scientists ®. 

SciSIP COMMUNITY 

The SciSIP, led by the National Science Foundation with critical input from Pro- 
gram Officer Julia Lane has made tremendous progress in spurring a group of schol- 
ars to pioneer studies in the science of science and innovation policy. For some of 
these scholars, this represented an increase in their commitment to a field in which 
they already had an interest. For others, SciSIP was a new departure and an oppor- 
tunity to move into a new and burgeoning field of great policy relevance and with 
significant intellectual challenges. The time is now ripe to move from funding of in- 
dividual researchers to extending and emphasizing the SciSIP community. A strong- 
er scholarly community — once established — will provide a number of critical bene- 
fits. It will be in a position to design and implement its own common pool re- 
sources and data sharing infrastructure to ensure that research methods, data and 
analytic tools are widely and effectively shared among scholars. At the moment 
there only a limited data-sharing infrastructure: the STARS program represents a 
key effort to gather new data, however many studies rely on complex historical 
datasets that incorporate rich and varied data sources but which are not shared 
across the community. While issues of confidentiality do arise, it is imperative that 
we follow the lead of the scientific communities we study and build a more effective 
infrastructure, norms and rules for data exchange and reuse ®. 

Community building will also enable a richer interchange across scholars whose 
disciplinary training and identify lies in different areas. At the present time, my 
perspective on SciSIP is that there exist various sub-communities largely within dis- 


distributional approach would enable SciSIP scholars to assess the impact of policies on 
numerous dimensions: researcher and institution status, nature of the researchers’ institution 
(university, private firm, government lah, etc.); researcher cohort; collaboration type (e.g., within 
vs. across institution, state, country, and/or field); basic vs. applied research; journal status; arti- 
cle breadth (multiple subjects vs. single subject); journal reputation (“impact factor”); and net- 
work characteristics. 

^ Some of my own work has examined the theme of gender in scientific research. In “An Em- 
pirical Study of Gender Differences in Patenting among Academie Life Scientists” (Ding, Murray 
& Stuart 2006) we show that for over 4,000 life science faculty, after accounting for the effects 
of productivity, networks, field, and employer attributes, the net effect of gender remains: 
women patent at 40% the rate of comparable men. Other research in this spirit includes Ding 
et al. (2009). 

®See Murray and O’Mahony (2007) for a detailed examination of the need for incentives for 
disclosure, reuse and accumulation in different knowledge communities and how these incen- 
tives are provided. 
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ciplinary silos who communicate but with little exchange across these traditional 
boundaries. For example, those who take an economics oriented approach gather as 
a community under the rubric of the Innovation Policy Working Group of the Bu- 
reau of Economic Research Productivity Program (including the Summer Institute 
Innovation Policy and the Economy activities). Not surprisingly however, this is not 
a forum in which sociologists, historians of science and technology or science and 
technology studies (STS) scholars share their research. In sociology there are few 
if any systematic gatherings of scholars with science policy interests and SciSIP re- 
searchers from STS and organizational behavior share similar concerns. One strong 
recommendation I have is for the NSE SciSIP program to fund the establishment 
of a “knowledge hub” that can orchestrate annual or biannual research meetings for 
interested SciSIP scholars. As I outline below in my comments on structuring 
SciSIP education, an effective cross disciplinary hub (that could be modeled on the 
Consortium on Cooperation and Competitiveness (CCC)) with governance from fac- 
ulty from a number of key universities and rotating responsibility for cross-univer- 
sity research meetings and some (limited) cross-university doctoral training. Such 
a forum should also enjoy strong input from the NSE but overall would be most ef- 
fective if it was organized with “bottom-up” support from faculty rather than man- 
aged directly by the NSE or other agency. 

Building stronger linkages between the SciSIP research community and the com- 
munity of science policymakers is another key pillar of the broader SciSIP commu- 
nity that remains to be constructed. At the present time, there is limited awareness 
of the key findings of SciSIP research among policymakers, and SciSIP scholars 
have only been engaged in a limited way in recent debates over key changes in 
science policy. For example, in the recent discussions over the role of innovation 
grand Challenges there was very little scholarly input from the SciSIP community; 
many prize and challenge designers and implementers were involved but there was 
little or no discussion of the tradeoffs associated with the use of challenges and the 
characteristics of the most effective problems that might be solved using challenges 
(and whose which are less likely to be tractable with this incentive system). Build- 
ing stronger links to the policy community is a long-term task that starts with the 
education of a new generation of policy makers to become critical consumers and 
co-producers of SciSIP research. However in the short run, links could be estab- 
lished with different government research funding agencies through a series of tar- 
geted workshops that bring policymakers, agency employees and SciSIP researchers 
to focus either on the issues, problems and successes of a particular agency or to 
focus on cross-cutting issues of mutual interest. This is likely to require sustained 
engagement through a series of regular meetings and dialogues in order to build up 
trust, mutual respect and an appreciation of the problems and opportunities that 
our nation’s research agencies, researchers and policymakers confront and the tools 
and insights that might guide them going forward. 

SciSIP EDUCATION 

Education is a critical element of the SciSIP agenda and should be a central pillar 
of SciSIP going forward. To date the program has focused largely on research and 
establishing a community of scholars among established academics. There is a 
pressing need to determine the best mechanisms through which to build up the edu- 
cational aspects of SciSIP and to fund this education. The challenge of SciSIP edu- 
cation can be considered along two dimensions — education of producers of SciSIP re- 
search and education of consumers/practitioners of SciSIP research. 

PRODUCERS 

The educational requirements of SciSIP researchers are intensive; the approach 
requires strong disciplinary foundations in the social science. These must then be 
complemented by three other elements: theory, phenomenon, data/empirics: 

• Theory: A perspective of the theoretical foundations that ground our under- 
standing of the behavior of scientists, the scientific community, and scientific 
progress (these can include a microeconomic approach based on under- 
standing incentives, the role of control rights etc. as well as a sociological 
focus on norms and practices or a psychological view) 

• Data/Empirics: Strong data and empirical skills specific to science and science 
policy. SciSIP is grounded in a belief that while every scientific research 
project is different, systematic data gathering, the use of both large-scale 
analysis (with publication, patent, citation, collaboration data) and granular 
field-data, and careful empirical design will enable scholars to draw causal in- 
ferences regarding the impact of specific policy levers (at the national, re- 
gional, agency, university and lab level) on scientific productivity and impact. 



46 


Therefore education must give researchers the ability to identify, gather and 
analyze such data 

• Phenomenon: A deep appreciation for the nature of scientific work and for the 
ways in which particular interventions in scientific progress have shaped pro- 
ductivity, impact or direction. This is challenging for scholars without a sci- 
entific training but is essential if scholars are to find the most effective re- 
search settings for their studies and if they are to make their work relevant 
to scientists and to science policy practitioners. 

The education of the “producers” of SciSIP research is a critical challenge that 
should be a high priority for the SciSIP community. Specifically, we must strength- 
en the education of PhD students who will become the leading scholars in the field 
developing the research agenda, pushing forward and filling research gaps and pio- 
neering new methods for the scientific and rigorous analysis of science and innova- 
tion policy. The skills needed to push this agenda forward are two-fold — first a 
strong disciplinary grounding in the “home’ discipline — economics, sociology, social 
psychology etc. and second, an in-depth understanding of the theories, data/ 
empirics, and the phenomenon (as outlined above). 

Establishing PhD “SciSIP fieid concentrations” within traditionai dis- 
cipiinary PhDs: In my opinion, it is not fruitful to try and establish a new dis- 
cipline within universities termed the “science of science and innovation policy”. In- 
stead I believe that it would be extremely valuable establish a “SciSIP field focus” 
within a variety of PhD programs within traditional disciplines including economics, 
sociology, public policy etc. At the present time, Public Policy schools are offering 
PhD degrees with a S&T policy focus. However, the promise of building a “science” 
of S&T policy is to extend the intellectual community well beyond the usual confines 
of policy analysis and ground the empirical and theoretical study of scientific pro- 
ductivity and impact in economics and sociology, as well as psychology and other 
adjacent disciplines. Therefore, as a complement to S&T Policy PhD education in 
Public Policy Schools it is critical to establish the field of “SciSIP” within the tradi- 
tional education of PhD social scientists within their traditional departments. [It is 
worth noting that this is not an effective educational path for those from a scientific 
background to move into SciSIP. To do so requires a switch into a social science pro- 
gram to learn the foundations of the particular social science discipline followed by 
a SciSIP field focus]. 

Let me illustrate the proposal of a “SciSIP field focus” with the case of economics: 
Building a “SciSIP field focus” within economics would involve establishing a suite 
of courses and educational materials at a small number of leading departments (who 
could share materials, exercises, data etc.). This could then be complemented by 
educational ‘bootcamps’ which would bring these PhD students together (from across 
schools) in a common forum to build their skills, build community and hear from 
leading SciSIP scholars. Such an approach would mirror the development of entre- 
preneurship as a field of study within economics — an area that was pioneered by 
the “Entrepreneurship Bootcamp” funded by the Kauffman Foundation and taught 
at the National Bureau of Economics. The National Bureau of Economic Research 
has played an important role in coalescing much of the activity around education 
in the economic foundations of entrepreneurship through the Entrepreneurship 
Working Group now part of the Productivity Program. This has enabled vibrant 
cross-school collaboration not only on research but also teaching. At the PhD level 
this has helped to build up and educate a community of young scholars within eco- 
nomics departments and management schools who now have additional training al- 
lowing them to pursue this field within their discipline. 

Hub and Spoke Approach: To build a strong and effective SciSIP-oriented PhD 
educational program will require using Federal education funding to actively seed 
a “SciSIP” field focus within at least 4 to 5 schools per disciplinary area (with at 
least two disciplines represented) — the Spokes. This should be supplemented by 
funding to develop an effective SciSIP ‘Hub’ for PhD education. The SciSIP Hub 
would coordinate activities that encourage coordination across these educational ef- 
forts, community building activities for the students involved, and the community 
“Bootcamp”. One model to develop and effective SciSIP ‘Hub’ for PhD education is 
the Consortium on Cooperation and Competitiveness (CCC) which “links together 
scholars interested in long-run performance of U.S.-based companies and institu- 
tions” but with a recent focus on PhD-level education, training and community 
building among PhD students from a number of programs (based mainly within 
leading Business Schools) with the involvement of academic faculty. As they de- 
scribed, “No single U.S. university or graduate school contains a “critical mass” of 
scholars from diverse disciplinary backgrounds concerned with issues that are pri- 
mary to CCC. Accordingly, the network structure of the Consortium is a significant 
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source of strength.”'^ A similar argument can be made with regards to the SciSIP 
agenda, suggesting that a similar consortium could be invaluable in advancing the 
PhD education and the scholarly community.® 

CONSUMERS 

The consumers of SciSIP research include Science and Technology policy makers 
as well as scientists and engineers at different stages in their education. Each of 
these groups could benefit from a deeper understanding of the results of SciSIP re- 
search. In particular, it should be a high priority of the SciSIP community to ensure 
that the SciSIP agenda is well understood within S&T Policy education; S&T Policy 
graduates are key stakeholders in the SciSIP research community will be leading 
consumers of our research, and partners in future research design and implementa- 
tion. 

Science & Technology Policy Masters Education: As students with Masters- 
level education in science and technology policy move out into the policy community, 
into research-based public policy organizations, and into the funding agencies that 
are the subject of much SciSIP analysis, they should be educated to be critical con- 
sumers of SciSIP research and to be co-producers of that research in partnership 
with academics. Much SciSIP research is relatively new and involves novel methods 
that are highly technical in nature and are not always taught to public policy re- 
searchers. Therefore, SciSIP has not yet been incorporated as a central pillar into 
the S&T Policy curriculum. For example, I supervise a number of MIT Technology 
and Public Policy students each year and find that they do not have an extensive 
and thorough grounding in the SciSIP approach, methods and results. Nonetheless, 
the students are quick to learn and start to use this approach in the course of their 
thesis work. However, it would be more effective to do this in a more programmatic 
fashion. 

NSF therefore has an important opportunity to work with a number of leading 
S&T policy programs around the country to develop a curriculum for education in 
the imperative, methods and results of the SciSIP agenda. This will require a dis- 
tinctive training from that provided to PhD social scientists for a number of reasons. 
First, these students can be expected to have less grounding in the data-oriented 
empirical methods that are common in SciSIP research. The focus should be on un- 
derstanding the empirical approach and critiquing its validity and the robustness 
of findings rather than on replicating studies. Second, it is critical to share an un- 
derstanding of the research design of SciSIP projects particularly those that are 
based on careful analysis of policy changes, policy experiments and other studies 
with a thoughtful counterfactual basis. This is a methodological approach that has 
been pioneered within SciSIP (as noted above) and is a critical element in the edu- 
cation of S&T Policy students. A greater understanding of the SciSIP approach will 
enable higher levels of collaboration between researchers and policy makers in the 
future. In particular, it has the potential to seed a higher willingness to work col- 
laboratively with scholars to design and analyze policy experiments with the goal 
of increasing our understanding of the impact of specific policy interventions on sci- 
entific progress 

Education of Scientists & Engineers: As has long been recognized in our anal- 
ysis of scientific productivity, faculty and students engaged in leading-edge research 
in science and engineering play an important and distinctive role in shaping the 
productivity and direction of their laboratories. Indeed the organization and direc- 
tion of large and increasingly complex research laboratories with collaborators that 
cross disciplines, cross universities, and often cross national boundaries is a 
daunting task. Nonetheless, we provide limited education to our science and engi- 
neering colleagues to guide them in this challenging activity. Offerings for scientists 
and engineers during undergraduate and graduate education are limited. As we de- 
velop new knowledge regarding the factors shaping research group productivity and 
the role of lab leaders in this productivity, it provides another opportunity for the 
National Science Foundation together with other leading funding agencies to work 
to provide such education. Effective education for scientists and engineers would in- 
volve three elements: 


http: ! / businessinnovation. berkeley.edu / cec.html 
®The CCC was funded by an initial endowment of $500,000 in May 1988 by the Walter and 
Elise Haas Fund. It has also been supported by grants from the Alfred P. Sloan Foundation 
in New York, the Smith Richardson Foundation, the Pew Foundations, the Ford Foundations, 
and the Herrick Foundation. From 1990—1995, the Sloan Foundations was the primary funding 
source for the Consortium. At the current time, the funding for doctoral activities is largely pro- 
vided by individual schools supporting their students and hosting the event and by the 
Kauffman Foundation. 



48 


• Teach science and engineering undergraduates about the role of science 
and technology in society and the economy and given them a broader perspec- 
tive on their technical education by highlighting the role of S&T policy. Focus- 
ing on the results of SciSIP oriented research will emphasize the importance 
of systematic, rigorous and data-driven approaches to policy, institutions and 
organizations. This will also provide them with tools to guide them in their 
subsequent careers, since they will run into the problems of the science and 
technology at every stage of their careers. 

• Provide PhD students with short courses regarding the ways in which their 
research can be more productive and have a more rapid impact on society and 
the economy based on SciSIP findings. Focus on the key interventions in the 
process of knowledge production (according to the SciSIP framework) — gov- 
ernment policies, regulations etc., university policies and practices, organiza- 
tional choices. Make this relevant through a focus on the career choices they 
will have to stay within academia, move into business or focus on policy. For 
those staying at the bench (in academia or industry) examine how to maxi- 
mize productivity and impact using the results of SciSIP research — organiza- 
tional choices they have available, the role of incentives in research teams, 
the most effective collaborative processes they can use, etc. Highlight the key 
processes involved in shaping commercial impact including entrepreneurship 
and technology transfer and the SciSIP results on how these are most effec- 
tively deployed. Finally highlight the key role of policy in shaping some of 
their choices. A program of this type has not, to my knowledge been devel- 
oped systematically for PhD students. This could be done in conjunction with 
other career-oriented activities provided by the NSF and other funding agen- 
cies to recipients of PhD grants and Fellowships. 

• Educate science and engineering faculty to have a deeper understanding 
of how they can achieve greater productivity and impact, based on the sys- 
tematic, evidence-based results of SciSIP research by running short courses 
at the university level (perhaps for new faculty), examining the organizational 
and institutional activities that they could undertake to increase the produc- 
tivity and impact of their laboratory. This could be incorporated into existing 
efforts on grantsmanship, communications etc. Possible topics could include 
two dimensions: factors shaping productivity including lab organizational 
choices, lab size choices, and collaborative models and factors shaping impact 
including patenting, technology transfer, materials sharing, networking com- 
munication etc. Such an approach would provide a platform for sharing the 
findings of SciSIP research with academic researchers while at the same time 
having an on the ground impact on the productivity of investments in re- 
search. Finding a possible funder of such an initiative would allow for key 
educational materials to be developed. The participation of key scientific soci- 
eties in this activity would also expand the set of stakeholders in the SciSIP 
agenda. 
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Chairman LiPiNSKl. Thank you, Dr. Murray. Dr. Teich. 
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STATEMENT OF ALBERT H. TEICH, DIRECTOR OF SCIENCE & 

POLICY PROGRAMS, AMERICAN ASSOCIATION FOR THE AD- 
VANCEMENT OF SCIENCE 

Dr. Teich. Thank you, Chairman Lipinski, Ranking Member 
Ehlers, other Members of the Subcommittee. Thank you for the op- 
portunity to testify at this hearing today. I am A1 Teich, and I am 
the Director of Science and Policy Programs at the American Asso- 
ciation for the Advancement of Science. 

As you know, AAAS and I myself have been deeply involved in 
science and innovation policy for many years. Although this has 
been an active field of research at least since the 1960s, and it has 
produced a large body of literature and a substantial number of re- 
searchers, there is a feeling that the results of this work are not 
widely known or used among those who actually make science and 
innovation policy. 

This was behind the frustration of Dr. Marburger in his speech, 
which led to the establishment of the NSF SciSIP Program. The 
SciSIP Program has a unique mandate to couple advances in fun- 
damental knowledge about processes of scientific discovery and 
technological innovation with issues of relevance to policy makers. 
Among the features that differentiate the SciSIP Program from its 
predecessors is the fact that it is not just supporting individual re- 
search grants, but it is attempting to build a community of practice 
among researchers and to connect that community with potential 
users of the research, practitioners in the Federal Government. 

AAAS has played an active role in building this community of 
practice through a workshop in 2009 that brought researchers to- 
gether to learn from one another. In that workshop, we saw how 
SciSIP researchers reflect distinct disciplinary traditions that can 
inhibit productive interdisciplinary dialogue. Even in this not-very- 
large field, they can’t always talk to one another. They may ask dif- 
ferent questions, use different theoretical frameworks that employ 
different methodologies even when they may address seemingly 
similar topics. 

At the same time, because of the academic reward system, 
SciSIP researchers, like many other researchers, seldom speak in 
terms that policy makers find directly useful. And as one speaker 
said at the 2009 AAAS workshop, policymakers are confronted with 
a Babel of tongues which leads them to ignore the experts and turn 
to other sources of information and advice. Next month AAAS will 
convene another workshop with NSF support. In that one we will 
try to connect researchers with customers in the government. We 
hope that that workshop will serve to allow the two communities 
to better understand each other’s needs and expectations. 

While projects like the AAAS-NSF-SciSIP workshops are an im- 
portant step in building a community of practice, there is more that 
can be done. Here are a couple of ideas just as food for thought. 
Regarding research, researchers tend to communicate directly with 
their peers by journals and conference presentations in order to 
gain recognition in their fields. But few policy makers read those 
journals or attend those conferences. We need to find ways to en- 
courage SciSIP researchers to communicate with policy makers, ei- 
ther directly or through the media, and to be rewarded and not pe- 
nalized by less policy-oriented peers in their fields for doing so. 
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On the teaching side, although many of the university programs 
that provide graduate training in science and innovation policy are 
interdisciplinary, the training they provide is not always responsive 
to the needs and priorities of policy makers. It might be useful to 
strengthen ties between researchers and policy makers by engaging 
policy makers in helping to develop and review curricula as well as 
engaging them in teaching as adjuncts or guest lecturers. Some 
schools already do this. Others would do well to follow their lead. 

Beyond education, there is another mechanism for promoting 
greater mutual understanding between researchers and policy 
makers. It is people transfer. One approach might be to create a 
program to give SciSIP researchers the opportunity to work in gov- 
ernment for perhaps a year. Providing SciSIP researchers an op- 
portunity to work in a policy making setting for a while, as we do 
for scientists and engineers in our Congressional Fellows Program, 
would allow them to gain firsthand knowledge regarding the needs, 
priorities, and modes of operation of the potential users of their 
work. Like our workshops, this hearing is an opportunity for the 
science policy community to hear from you, as policy makers, what 
research questions you believe SciSIP researchers should be ad- 
dressing. I look forward to the Q&A as an opportunity to exchange 
ideas on that subject. 

[The prepared statement of Dr. Teich follows:] 

Prepared Statement of Albert H. Teich 

Chairman Lipinski, Ranking Member Ehlers and members of the Subcommittee, 
thank you for the opportunity to testify today on the evolving subject of the Science 
of Science and Innovation Policy. 

The American Association for the Advancement of Science (AAAS) is the world’s 
largest multidisciplinary scientific society and publisher of the journal Science. The 
association, which celebrated its 162nd birthday earlier this week, encompasses all 
fields of science, engineering, mathematics, biomedicine and their applications. For 
more than thirty-five years, AAAS has demonstrated its commitment to and involve- 
ment in science policy issues with projects and activities such as the annual AAAS 
Science and Technology Policy Forum, the Science and Technology Policy Fellows 
Program, more recently with our Leadership Seminar in S&T Policy, and — most di- 
rectly relevant to this hearing — our joint project with the National Science Founda- 
tion on the Science of Science and Innovation Policy (SciSIP). We have served the 
academic science policy community by publishing the first Guide to Graduate Edu- 
cation in Science, Engineering and Public Policy (known as the SEPP Guide) in 1985 
and maintaining it as an online resource to the present day. 

Background 

From one perspective the Science of Science and Innovation Policy is not entirely 
a new field. Since the 1950s — and probably earlier — economists, sociologists, polit- 
ical scientists and others interested in public policies for science and technology 
have sought ways of measuring the value of research investments. Research articles 
on topics such as measuring Return on Investment (ROT) from research and devel- 
opment (R&D), national innovation systems, and comparisons of state and inter- 
national standings have been published for many years. Government tools such as 
the 1993 Government Performance and Results Act (GPRA) and the more recent 
Program Assessment Rating Tool (PART) as well as their programmatic forebears, 
have attempted to quantify the value of government investment in various pro- 
grams, although they have found R&D programs more difficult to assess than oth- 
ers. 

In the 1960s, the National Science Foundation (NSF) supported the development 
of research and graduate education programs in science and technology policy in a 
number of universities. During the 1970s, it created the R&D Assessment and R&D 
Incentives programs, which funded research on some aspects of S&T policy in uni- 
versities and non-profit institutions. In addition to the SciSIP program, the Founda- 
tion currently funds research in science policy and related areas through a number 
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of programs, including the Science, Technology, and Society Program in the Direc- 
torate for Social, Behavioral, and Economic Sciences, and the Division of Science Re- 
sources Statistics, which has long provided data and analysis of importance to 
science policymaking. 

The current Science of Science and Innovation Policy endeavor is unique, however, 
in its focus on drawing this research together into a systematic, coherent body of 
knowledge that can be brought to bear directly on national policy decisions. The Na- 
tional Science Foundation’s SciSIP program is engaging the science policy commu- 
nity in research in theory, methods, models, and data development along four broad 
themes — workforce issues, innovation ecosystems, outcome measures, and data in- 
frastructure. The program has an explicit mandate to couple advances in funda- 
mental knowledge about processes of scientific discovery and technological innova- 
tion with issues of relevance to policymakers. As a field of research, the Science of 
Science and Innovation Policy has essentially been raised in relevance from a large- 
ly academic discourse to a field with a potential national impact. 

Science and technology policy research can have and has had a positive effect on 
national policy decisions. R&D data analyzed and reported by NSF, as well as by 
AAAS, for example, has provided a roadmap for decades for policymakers such as 
the Members of this Committee as a guide for crafting the federal R&D portfolio. 

As the NSF SciSIP program is still quite young and has been awarding grants 
for only a few years, we believe that it is premature to expect the results of that 
program’s research to be incorporated into national policy decisions. Furthermore, 
the results of any science and technology policy research — whether within or outside 
SciSIP — must still run the gauntlet of the policy process. 

In other words, simply because the research has been done, does not mean that 
it will be used. As helpful as the AAAS R&D budget analysis may be to its users, 
policymakers still make decisions based not only on research and analysis, but also 
on constituent needs, economic and political considerations, public opinion, and their 
own perspectives on national priorities. The same goes for studies that measure the 
effectiveness of federal programs. Politics is not a contaminant in the policymaking 
process. It is, after all, the essence of a democracy. 

One way that policymakers can increase the likelihood that SciSIP research be 
used to inform the design of effective federal programs and the management of fed- 
eral research investments is to conceptualize and design research that both ad- 
vances knowledge in a discipline and answers specific questions relevant to policy. 
Some examples of such research topics are given in the NSF SciSIP program solici- 
tation: 

• examinations of the ways in which the contexts, structures and processes of 
science and engineering research are affected by policy decisions, 

• the evaluation of the tangible and intangible returns from investments in 
science and from investments in research and development. 

It should be pointed out that science and technology policy research is just as un- 
predictable as basic research in physics, chemistry, or life sciences, and decision 
makers must take into consideration the fact that some studies may yield unantici- 
pated results and that some may serve long-term rather than short-term needs. It 
is important to ensure that an effective SciSIP portfolio balances research that re- 
flects short-term and long-term policy interests. 

Among the features that differentiate the SciSIP program from similar, past ef- 
forts, is its focus on building a community of practice among researchers in the 
many disciplines engaged in the study of science and innovation policy and its con- 
scious effort to build bridges between this community and the practitioners in the 
federal government. Previous programs to support science and technology policy re- 
search have always focused primarily on providing grants to individual principal- 
investigators. 

AAAS has played an active role in building this community of practice. We orga- 
nized a workshop of the grantees from SciSIP’s first and second rounds (FY 2007 
and 2008) of awards to further construct this community. The outcome of that work- 
shop was a report, titled, Toward a Community of Practiced Next month we will 
convene a second workshop to continue building a community of practice by con- 
necting the researchers with potential users of their results in the federal policy 
community. 


1 Albert H. Teich and Irwin Feller, Toward a Community of Practice: Report on the AAAS- 
NSF Grantees Workshop, March 24-25, 2009 (Washington, DC: American Association for the Ad- 
vancement of Science, August 2009). Available online at http: I / www.aaas.org I spp / scisip/ 
scisip-report.pdf. 
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There are challenges to building this SciSIP community of practice. A sizable 
group of researchers working on current projects as well as a large body of literature 
already exists. To an important degree, these individuals and this literature reflect 
distinct disciplinary traditions that can inhibit a productive interdisciplinary dia- 
logue. These disciplinary clusters may ask different questions, draw upon different 
theoretical frameworks, and employ different methodologies and analytical models 
even when they may address seemingly similar topics (e.g., diffusion of innovation). 
Sometimes it seems they even speak different languages. 

At the same time, as these researchers speak to an audience of their peers — albeit 
within their disciplines — they often do not speak a language that policymakers un- 
derstand or find useful. A concern expressed at the first AAAS SciSIP workshop was 
that policymakers would be confronted with a “Babel of tongues” which would lead 
them to ignore the experts and turn to other sources of information and advice. 

Another challenge is the fact that not all SciSIP researchers have experience 
working at the interface between academic research and federal policymaking. Some 
lack an understanding of the user community and who the policymakers are, what 
information or datasets they might require, or what other information they might 
need to know in order to effectively address national policy priorities. This is not 
to imply that these researchers are not familiar with the organization of government 
or the legislative process. Rather, it has more to do with the subtleties and nuances 
of the “game” and having an insider’s perspective on the complex policy questions 
that decision-makers face and the interplay of interests that often shapes the debate 
over science and innovation policy. 

The AAAS project is an effort to build these necessary relationships and to help 
SciSIP researchers and policymakers speak each other’s language and better under- 
stand each other’s needs and expectations. The goal is not to build a grand over- 
arching theory of science and innovation policy, but to seek convergences among 
findings and a higher degree of understanding within the community about new 
perspectives and paradigms regarding science and innovation policy. It is to build 
a more interdisciplinary approach with an eye towards practical application by prac- 
titioners. 

This community of practice is intended to assist individual researchers or teams 
of researchers by enlarging the set of variables and/or relationships that they con- 
sider in their work. It provides an opportunity to expose research findings to a wider 
set of critical perspectives and allows researchers to consider how their findings may 
relate to other disciplines and research findings in other areas. 

As you know, the NSF initiative in the science of science and innovation policy 
stemmed from a sense that the body of science and innovation policy research does 
not seem to be very widely known or used among those who actually make policy 
in these areas. The AAAS SciSIP project is intended to facilitate interaction between 
relevant federal agency representatives and the growing community of SciSIP re- 
searchers, to help the agency representatives learn about emerging theories and 
models, and to connect research results with policy issues. At the same time, SciSIP 
researchers should be able to learn from the user community about their policy pri- 
orities and needs, which can help shape the direction of future projects. 

While the SciSIP program and projects like the AAAS-NSF SciSIP workshop are 
an important step in building a community of knowledge and a strong foundation 
between research practitioners and policymakers, there is more that can be done. 

Communication: As noted earlier, researchers addressing questions of science 
and innovation policy have tended to direct their work to colleagues, peers and oth- 
ers within their core discipline. This includes presentations at professional associa- 
tions and conferences, and publishing in specialized journals (e.g.. Research Policy, 
Social Studies of Science). This is quite understandable in view of the reward struc- 
ture of academia and desire on the part of scholars in this field and others to gain 
recognition from their peers. Relatively few policymakers read such journals or at- 
tend academic conferences with any regularity. One could approach this problem in 
two ways: One approach would be to encourage policymakers to read these journals 
and/or attend more academic conferences. Given the constraints of time and energy 
they face, this seems unlikely to work. Alternatively, SciSIP researchers might seek, 
in addition to their regular publication outlets, opportunities to reach out to policy- 
making community either through themselves writing for those publications that 
policymakers do read or by cultivating opportunities to have their work reported in 
popular media. 

Education and Training: This “clustering” of a narrow core discipline has not 
only worked its way into the presentation of information, but in the education and 
training of students studying science and innovation policy that only encourages 
self-organization of a research area. Although the AAAS SciSIP program may help 
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in encouraging the development of a more interdisciplinary curriculum, it isn’t the 
central goal of the project. 

As the committee has noted, there are about 25 U.S. universities that offer grad- 
uate education in science, engineering and public policy. There is no central organi- 
zation for these programs and do not share a common curriculum or even a common 
nomenclature. The AAAS Guide to Graduate Education in Science, Engineering, and 
Public Policy mentioned earlier lists programs such as Science Policy; Technology 
Policy; Science and Technology Policy; Science, Technology, and Public Policy; and 
Engineering and Public Policy. In addition, many programs in Science and Tech- 
nology Studies (STS) include a policy component, and some programs in public ad- 
ministration and public policy provide for a science and technology concentration. 
Furthermore, these graduate programs can be administered within different aca- 
demic departments: Schools of Engineering, Public Administration, International Af- 
fairs, etc. Some programs allow for students to take coursework outside the tradi- 
tional curriculum in other tangential fields (e.g., law), while other schools do not. 

Many of the graduates of these programs have gone on to very successful careers. 
Nevertheless, it might be useful to have people from the policy community — the po- 
tential users — involved in reviewing the curricula of these programs as well as en- 
ga^ng in teaching as adjuncts or guest lecturers. This is obviously easier for univer- 
sities in the Washington, DC, area to do than for those in other regions and some 
institutions in this area do it regularly to good effect. But it is worth the effort and 
expense for all. 

Fellowships: Another potential mechanism for promoting cross-fertilization of 
ideas and greater understanding of the policymaking community’s needs, is to create 
a Fellowship program for SciSIP researchers to work in government for one year, 
similar to the AAAS Science and Engineering Policy Fellowship that allows sci- 
entists an opportunity to work at a federal agency or in a congressional office or 
committee. Intergovernmental Personnel Act appointments could also be used for 
this purpose. Providing science and policy researchers and/or graduate students an 
opportunity to work in a policy office of the federal government would allow them 
an opportunity to learn first-hand the language, needs, and priorities of an agency, 
department, or congressional committee. 

Conclusion. 

I would like to thank the Members of the Subcommittee for holding this hearing 
and for their interest in the SciSIP program and the area of science and innovation 
policy research. I look forward to working with your staff as we prepare for the next 
AAAS workshop. Like our workshops, this hearing is an opportunity for the science 
policy community to hear from you, as polic 3 unakers, what research questions you 
believe SciSIP researchers should be addressing. I look forward to the Q&A as an 
opportunity to exchange ideas on that subject. 
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Washington University, the State University of New York, and Syracuse University. 
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Technology and Society; the Program Committee for the 5th EuroScience Open 
Forum (to be held in Dublin, Ireland, in 2012) and the Council of Advisors for Re- 
search and Innovation Strategy of the National University of Singapore. 

Teich speaks frequently before audiences in the U.S., as well as Europe and Asia. 
He has appeared on National Public Radio, CNN, C-SPAN, as well as various other 
electronic media and has been quoted in numerous print media, including The New 
York Times, The Washington Post, National Journal, The Chronicle of Higher Edu- 
cation, and CQ Weekly Report. 

Chairman LiPiNSKl. Thank you Dr. Teich. I am going to start 
questions. If you want to leave to get over to vote I think we have 
about four minutes left, probably, in the first vote. But I think this 
vote is going to last a long time. But we will — if the witnesses can 
come back afterwards — it is probably going to be about an hour 
though. I am not sure if any of you have — we will have to leave 
at that point, but — ^yes. Dr. Ehlers, yes. You have a suggestion? 

Mr. Ehlers. No, just a quick comment which shows the impor- 
tance of this topic and why we should come back if we can depend- 
ing on the votes. But I would simply observe that the current proc- 
ess in the Congress is that science policy is set by the Appropria- 
tions Subcommittees. Money controls everything and when they de- 
cide to give a certain amount of money to a certain project, that 
basically ends up being the decision. That totally ignores the input 
of other scientists and SciSIP’s folks who have a much greater in- 
terest. So something you can think about in the meantime is how 
that could be addressed without throwing out the Appropriations 
Committee entirely which is probably impossible. So I just wanted 
to mention that, and I hope you will have some brilliant ideas on 
how we could practically address that particular problem. My staff 
just informed me that 300 people have not yet voted, so we could 
probably walk over instead of running over. But I hope the votes 
don’t run too long. And I would be delighted if any of you would 
take on the challenge too and follow this. 

Let me add one quick last comment. When Newt Gingrich was 
here he wanted to double the funding in NIH, which did happen 
in the Appropriations Committee. I argued that we should have 
equal funding increases for NSF, treat all the sciences equally. He 
said we will do that one next. Well, unfortunately we lost the ma- 
jority and so the next — they were happy. But today I have heard 
Newt say in numerous speeches that one big mistake he regrets is 
not having increased NSF and the other hard sciences at the same 
time when he increased NIH. So let that be a moral note for all 
of you who hope someday to be the Speaker of the House of Rep- 
resentatives. Thank you very much. 

Chairman LiPiNSKl. I am hoping that you can make sure you 
spread that work amongst your colleagues before you leave. 

Mr. Ehlers. Yeah. 

Chairman LiPiNSKi. So you can help us — who really wants to 
make sure we get that done, get that done in the future. Well, you 
now have your homework assignment. You will have probably 
about an hour and we will be back. Hopefully sooner than that, but 
it is going to be at least 45 minutes I would say and I look forward 
to hearing your answers. I am most interested in how we really 
make these connections. Dr. Teich, I appreciate some of your com- 
ments. I would like to delve maybe some more into how we can, 
having been a — now as a political scientist, and talk about not — 
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policymakers don’t read the journals. Political scientists weren’t 
reading the journals because it didn’t really speak to them, much 
less the policymakers. But I would like to delve into that also some 
more, how we can improve that. But the Subcommittee will be in 
recess. 

[Recess.] 

Chairman LiPiNSKl. I call this hearing back in. I will now start 
the questioning. I understand Dr. Sarewitz has to leave at four 
o’clock so we will — each of us will get the chance to ask some ques- 
tion before you have to leave. So I will now recognize myself of five 
minutes and will begin with Dr. Sarewitz. 

You mentioned in your testimony that most of the data available 
for SciSIP analysis are input/output data, level of funding, number 
of graduate students, patents, et cetera, publications. And if you — 
data offer an incomplete view of societal value of S&T investments. 
So what would you suggest that we do to better characterize and 
measure the social outcomes of R&D? 

Dr. Sarewitz. Okay. Thanks for asking that. It actually — see, 
how should I put this — it — my answer will reflect a diversity of per- 
spectives here. I think we can all agree that the process — and Julia 
actually wrote about this wonderfully in Science Magazine — that 
the process that leads from R&D to a particular desired social out- 
come, for example more employment or better health, is extremely 
complex, with many different inputs into the process. But I think 
that measuring is one way to understand things but also very close 
case-based and textual analysis is another way to understand 
things. And my view is that the system is so complex that we are 
probably not going to come up with a big theory of how you can 
predict social outcomes from science and technology inputs. But we 
are going to be able to develop a number of principles that reflect 
our understanding of particular examples. 

So I think the kind of data that — and I wrote about this a little 
bit in my testimony — that we really need a kind of — data that we 
are lacking that would be very important is very granular case 
studies of both successes and failures in this full range of linkages 
from laboratories to social outcomes for a particular range of sci- 
entific priorities. And I think by doing that we will be able to elicit 
a set of general guiding principles that can help you guys distin- 
guish between policy decisions that make sense and policy deci- 
sions that don’t make sense. I guess I am a little skeptical of the 
idea that we will ever be able to actually predict with precision. 
But I think we can be a lot smarter about the basic set of assump- 
tions if we can develop some really close case studies from end to 
end, case studies that show great detail. 

Let me just quickly say, one, we are looking, for example, at Ari- 
zona State we are looking at the development of the solar power 
industry in Arizona, because obviously we have a lot of sun there. 
And so it is not — one of the important inputs of course is R&D into 
the solar power industry, but there are all sorts of local dynamics, 
from water availability, land use, obviously regulatory frameworks, 
all of those things are important and they are not generalizable. 

So while I think we can develop a very rich case study around 
solar in Arizona, I don’t think we should necessarily worry about 
a grand theory. So we should develop best practice case studies 
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looking very closely at the full process of leading from the R&D ac- 
tivities themselves to the societal outcomes. 

Chairman LiPiNSKl. Did anyone else want to — any of the wit- 
nesses want to comment on that? Have anything to add to that? 
If not I am — now I think about this in your answer, Dr. Sarewitz 
and I — do we have the data available right now? Would we need 
to do a better job of collecting data so that we can do this kind of 
research? The whole generalized ability of this is when you look at 
almost anything that is really a social process. I always go back to 
my days as a political scientist in trying to put together these theo- 
ries that will predict outcomes and the struggle with doing that 
and trying to make political science into physics. How much can we 
do here when we are talking about doing the SciSIP research, and 
what we can really glean from the data that we have? 

Dr. Sarewitz. So let me just say this. A diversity of perspectives 
is here and that is good. I mean, I think it is a rich field and it 
needs to bring lots of perspectives together, from the highly quan- 
titative model to the more case-based qualitative work. We need all 
of that. I think we know a lot. I think Dr. Teich’s point about the 
problems of communicating what we know is really important, and 
that thinking about how to communicate more effectively the 
things that we already know, for your benefit, is an essential part 
of it. And so two things need to go on simultaneously. They are — 
this field is really only just beginning to kind of get its legs. 

Dr. Murray talked about how she’s been doing it for a long time, 
didn’t know there was a field out there. I have been doing it for 
a long time as well, but more or less in small groups. So Dr. Lane’s, 
you k^now, efforts to create a community does two things. One is, 
it creates — it has created the intellectual momentum that we are 
going to need to move the field forward, but it also allows us to 
really collect what we know already, which I think is considerable, 
and present that, if we can figure out how to communicate effec- 
tively. I would be glad to talk about that a little bit, too, if you 
would like. 

Chairman LiPiNSKi. Well, let us come back. Right now I am 
going to yield back my time. I assume my time is up and I want 
to yield now to recognize Dr. Ehlers for five minutes. 

Mr. Ehlers. Thank you very much and I don’t have any ques- 
tions for you Dr. Sarewitz, other than to note that we produce 
weather today that is very close to what you have back home. We 
did put a little moisture in the air as well, so that is a little dif- 
ferent. 

Dr. Sarewitz. I wouldn’t be dressed like this either. 

Mr. Ehlers. That is true. But I appreciate you coming. I don’t 
have any questions for you that have not been either answered or 
explained already. But I would like to ask on the two ends of the 
panel. Dr. Lane, Dr. Teich, you both are quite familiar with the 
Congress and how it operates. Do you have any suggestions on 
what someone in the Congress could do to help educate our Mem- 
bers about the importance of science policy and what it should be, 
what it can do, what it cannot do, and any wisdom you could give 
us I think would be very helpful as we go forward in the Science 
Committee and try to — I hate to use the word modernize, but you 
know what I am talking about. Just try to get the workings of the 
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House of Representatives and the Senate to reflect reality, and 
what should be done about the Science of Science Policy and in par- 
ticular, what role science policy should have in guiding the Con- 
gress on the very difficult issues we have, particularly those relat- 
ing to funding. So we will start with you. Dr. Lane, and go to Dr. 
Teich, and also Dr. Murray if you have any comments on that. 

Dr. Lane. Well, thank you very much for that thoughtful ques- 
tion. I am not as wise in the ways of Congress as you, obviously, 
so this is very much in the spirit of the suggestion rather than an 
expert approach. One of the things that I think is most important, 
that will get Congress to understand the value of science invest- 
ments, is evidence. If there is clear evidence of the impact of 
science investments, on the four sets of dimensions — social, sci- 
entiffc, economic, and work force — that both has a qualitative as- 
pect, that is, that there are real people affected, and there are real 
advances that are made in the quality of life, but also quantitative. 
That is, when you can unambiguously say there were — this amount 
of investment led to a whole variety of different sets of outcomes, 
and that tracer is clear. I think that is what gets people in 
Congress’s attention, because obviously they are serving the Amer- 
ican taxpayer, and that is what the American taxpayer is inter- 
ested in ffnding out. 

Mr. Ehlers. Okay. Dr. Teich. 

Dr. Teich. Yeah, I think I would turn that around a bit and 
point out that it is really very much up to us in the SciSIP and 
science policy communities to communicate effectively with you in 
the Congress. You have so many messages coming at you from so 
many different directions that somehow, what we need to do is dif- 
ferentiate the kinds of information that we have, hopefully evi- 
dence-based. And we have to recognize that it is not the only influ- 
ence, the only thing that you have to take into account in making 
decisions. 

That decision — I was struck by something Chairman Lipinski 
said about making political science into physics. I started out, I got 
an undergraduate degree in physics and my Ph.D. in political 
science, and you know physics; in some respects physics is lot easi- 
er, you know. You start, you can — my freshman physics, you know, 
assume a frictionless plane. Okay, well you can assume a 
frictionless plane and it works in some respects. Assume a 
frictionless Congress and you know you have got nothing. It doesn’t 
make any sense. So there is a — politics isn’t neat. It is not, and 
data doesn’t always trump a lot of other factors that go into deci- 
sions. We have to understand that we have to communicate within 
that framework, and then I think it is up to you in the policy com- 
munity to make use of that. Best I could do on short notice. 

Mr. Ehlers. If we had a frictionless Congress, things might go 
better. Dr. Murray, do you have any wisdom to add to this? 

Dr. Murray. I am not sure it is wisdom. It is certainly a thought 
I have, is that — I think it is important for us to provide data that 
is meaningful. I think it is also important for us to think about 
studies that really show, again, sort of causal impact. So I think 
that there is some new work that has been funded by SciSIP and 
in other places where we can say, look, you know we did have a 
quite big shock to the system in terms of additional funding going 
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in quite rapidly through the Recovery Act, and some of the spend- 
ing in other countries that means very big shifts in research fund- 
ing allocation that have happened relatively quickly. And so I think 
we have a lot of opportunities to both study those things and also 
to marshal that evidence — because I think you could always go in 
and just say, we want more for science and technology, and every- 
body has heard that and of course we are going to say that. And 
so I think that coming in with evidence that says — when you get 
these shifts, both in level and distribution, real things happen, real 
differences, and outcomes happen. I think if we can marshal that 
evidence in a persuasive way, then I think we can be much more 
informed and are much more likely to be listened to. 

Mr. Ehlers. Okay. Well, those are very good comments. I worry 
a little bit about the Congress requiring a lot of evidence because 
you know many experiments don’t come out that well and the Con- 
gress would say, now — next time you come around, say, well, you 
know, you sold me on this project and nothing really good came out 
of it. And that is pretty hard to overcome. 

I really appreciate the ideas you have presented and the com- 
ments you have made today. And it has given me some new in- 
sight. And I really do think that we need more concentration on 
this not only in the Congress, but among the science policy commu- 
nity. And what I said several times earlier on about this was — I de- 
liberately said “Unlocking the Future” because I wanted someone 
in the future to write better, something better about science policy 
and something along the line of Vannevar Bush’s book which was 
probably — it could have been what we want today, but nevertheless 
he addressed a lot of issues that had to be taken into account. He 
himself was very different but very concerned about the fact that 
Congress did not pick up on a lot of his suggestions, and particu- 
larly one creating a different version of the National Science Foun- 
dation, but yet out of his work and his arguments, eventually, I 
think some ten years after he wrote the book, they did start estab- 
lishing the National Science Foundation. So even though he re- 
garded his work as a failure because the Congress didn’t pick up 
on it, eventually it did happen. 

So I encourage the science policy community to become very ac- 
tive, and frankly, also very aggressive in your addressing Members 
of Congress. It would not hurt at all if a few people from the 
science policy committee ran for Congress and got elected. I just 
had an experience on the Floor not 10, 15 minutes ago. Someone 
came up to me and had been present this morning at the Science 
Committee meeting and said Vern, what in the world are we going 
to do without you, because I had used my scientific knowledge in 
a number of statements. And I say well, I think, you know I don’t 
think I do that much. They will get along. But the matter of fact 
is there won’t be any scientists left on the Science Committee. And 
it is just helpful to be inside all the side discussions that are held. 
It is good to have someone there. 

So I repeat, as I have done with every speech I have given to 
every engineering or scientific group: run for Congress. We need 
more scientists in the Congress, and incidentally not just for the 
benefit of science, but most scientists are fairly clear thinkers on 
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issues and frictionless or not, and they have a lot to contribute to 
the operation of the governing bodies of this country. 

I would actually say I probably got — had much more impact at 
the state level because I was truly a rarity there. And most state 
governments don’t have the resources to have scientists on staff. 
And I had endless amounts of work to do trying to resolve things, 
such as resolving difficulties between optometrists and ophthalmol- 
ogists, or dealing with questions such as the foam installation that 
was the rage for awhile pumping into homes and now people are 
sick from formaldehyde fumes from that. These are issues no one 
in the state legislature was equipped to deal with, and I resented 
all the time I had to spend on it, but at the same time it was very 
useful to society. So spread the word, please, and thank you again 
for being here. I appreciate it very much. 

Chairman LiPiNSKl. Thank you. Dr. Ehlers. 

Mr. Ehlers. Chairman, I beg your pardon but I have a bill on 
the floor that has just been called up and I have to rush down 
there to speak on it, so my apologies. 

Dr. Teich. Mr. Ehlers, before you go I just want to say on behalf 
of AAAS, the science community and the SciSIP Community, we 
are going to miss you. Thank you for everything that you have 
done. 

Mr. Ehlers. Thank you very much. I appreciate that. 

Mr. Lipinski. Dr. Ehlers, we will assume I have your permission 
to continue here and wrap up. See as there was no objection from 
Dr. Ehlers I will — I was asking for your permission to continue on 
and to wrap up here. Thank you. Okay. You are still here, and 
frictionless. I will now recognize myself for five minutes. I — it is 
funny, the — talking about the assumptions and comparing physics 
to — or trying to make political science into physics. I had a col- 
league of mine in grad school in political science who was also, like 
myself, had a background in engineering before going to get a 
Ph.D. in political science, and he always would say that political 
science had physics envy and we were trying to be physics. Now 
it did not stop. Political scientists, and even focusing specifically on 
Congress, Congressional scholars did not — some were not afraid to 
make assumptions that wind up where they were talking about 
something that was supposed to be Congress but pretty much — 
very much not Congress anymore after all the assumptions that 
were made, and make all these assumptions that said, with this, 
we were dealing with a imaginary legislature, but then we are 
going to pretend like it is Congress. Hopefully that is not the type 
of work that is going on here in SciSIP. 

I want to make sure — one thing I wanted to ask — Dr. Murray 
talked about this, and I want to ask everyone about, if they have 
any more comments on this. Because I know Dr. Murray, in your 
testimony you talked extensively about it as training, you know, 
more people to be able to do this research in having programs that 
produce the type of, you know you, go into what she is talking 
about, Ph.D. programs, but we need to, in general, produce people 
who can do this, to do this work. And I know that people who are 
doing this, researchers in this field, are located — as I mentioned in 
my opening statement — in all kinds of different places. 
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Dr. Murray, I know you are in the business school. Are there any 
suggestions — I don’t know if there is anything else you wanted to 
add, Dr. Murray, or Dr. Teich, or if Dr. Lane would want to say 
anything about where we are right now in terms of programs that 
are producing researchers that can do SciSIP — where that is going. 
Are there programs such as this that are out there? If not, where 
are they coming from? Is this something that we need to, you really 
think we need to do more of and concentrate on, to found such a 
specific field like this, or can we get by with people coming from 
different fields. Is that the way to do it? So I just want to throw 
that question out there. As a, former political science academic I 
am interested, in you know, questions like this in terms of what 
we are doing out there in higher education. 

Dr. Lane. So I think that is a very interesting question. It is an 
important question. The main area — if you are going to train peo- 
ple in doing this kind of research, they are going to go into the field 
and do the kind — develop the kind of skill sets that we need. You 
want them to be able to get tenure. You want there to be able to 
be a career ladder. And the program, besides SciSIP, isn’t sufficient 
to support an interdisciplinary field in its own right. Nor is it, I 
think, possible to develop career paths for such a narrow set of 
skills. 

So I think what is important is to convince very smart people in 
economics, and sociology, and psychology and many of the other 
areas, that feel that science policy is a really important and inter- 
esting field, that they can bring their skill sets to bear on, to an- 
swer important science policy questions and that they can publish 
and get — advance within their own disciplines. So I think that is 
what is critical rather than trying to establish a separate field in 
its own right. I don’t think that is feasible given budget constraints 
and so on. So that is what we have been explicitly been trying to 
foster, to make it an intellectually challenging, exciting, and pub- 
lishable type of field. 

Chairman LiPiNSKl. You going to comment Dr. Murray? 

Dr. Murray. Yes, I think it is — I think that there are three dif- 
ferent constituencies for education. One is the Ph.D.’s, who are 
probably the producers of research. The other — and then there are 
the science and technology policy, typically Master’s students, who 
tend to go into policy roles who I think we need to educate to be 
better consumers [of this research] and also people who really un- 
derstand what we do and can help do it with us. And then finally 
there are, probably, the scientists and engineers themselves who 
could benefit from understanding some of this. We then become a 
sort of bottom-up constituency who can shape agencies and so on. 
I think on the Ph.D. side, I think Julia is exactly right. I don’t 
think you can have a new discipline of SciSIP. I think it is both 
too small, and, in fact, one of the great values of SciSIP is indeed 
the fact that people come from these other strong disciplinary foun- 
dations. I think what we do need to emphasize, though, is a seri- 
ous, a sort of a field focus. 

If you think of a Ph.D. in political science or economics, mostly 
there is a field. At the moment I don’t think many places already 
have a field focus in something that we would recognize as SciSIP. 
And I think that we could go a long way towards funding things 
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that would help establish that. You know, Ph.D. education re- 
quires, especially in something like this, you know, significant evi- 
dence, and teaching materials, and data sets, and things so that 
students can work on this, and so that we can effectively collabo- 
rate across a set of schools to really begin to develop material, 
share expertise. And then potentially bring the Ph.D. students to- 
gether as a community so that they recognize one another even 
though they are always going home and we know we are educating 
them to be hired by business schools’ economics departments and 
so on. 

So I think that there is an opportunity there as long as we make 
sure we know what we’re trying to accomplish, which is not a new 
discipline. I think on the science and technology Master’s side, I am 
less familiar with that because even though I do SciSIP research 
I don’t teach in the technology and policy program at MIT. But that 
in and of itself tells you something, which is that there is, I think, 
a little bit still of a disconnect — that the traditional science and 
technology policy programs have not necessarily sort of incor- 
porated SciSIP research into their teaching material. 

And so again I think that there is an opportunity to do some- 
thing about that. Not to insist that people do it, but to provide op- 
portunities to develop a really effective curriculum so that as peo- 
ple go into different — into their careers as policy makers, they un- 
derstand what we are trying to do, some of the methods, they know 
good SciSIP research from less good SciSIP research and they 
themselves can say oh, you know, we are doing something in our 
agency. We could actually run that as an experiment that could be 
studied. We could try two different ways of allocating funding and 
we could really do the analysis with real data. And I think if we 
could educate people to that level, we would have a much better 
interchange in the long run, and it would be a really — it would be 
a very vibrant community. 

Chairman LiPiNSKl. And Dr. Teich. 

Dr. Teich. Yes, well, a couple of things worth noting in response. 
First of all, there are and have existed for some time about 25 pro- 
grams in universities around the United States, and some outside 
the U.S. in addition to those 25 . They have provided graduate edu- 
cation in science — in what we have called science, engineering and 
public policy, and which overlaps quite substantially with what we 
now call SciSIP. We had many years ago we published a guide, an 
old-fashioned paper type guide. We now have a website on the 
AAAS website that links to all of these programs which could help 
people find them. 

I don’t see this as a discipline either. As was said a moment ago, 
it is a — my analogy is that it is more like a field of, let us say, area 
studies in which, like Latin American studies, for example, or Afri- 
can studies, it is a field in which many different disciplines con- 
tribute to an understanding of what is going on in this business. 
So that is one thing that I wanted to mention. 

Another thing is that there is — there are a lot of young people 
who are very interested in this, and we need to encourage them. 
There is an organization — an international non-profit — it is incor- 
porated as a 501(c)3 called Triple Helix, Inc., which has about 500 
students from many universities, prestigious universities around 
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the world, which provides an opportunity for students to educate 
themselves about the relationships between science and society, in 
ethics, business, and law. They actually publish an undergraduate 
journal, which — a couple of people from AAAS’s staff serve on their 
Board of Advisors. They also have a poster session at the AAAS 
Annual Meeting. 

And then there is a group called the Science and Technology, or 
STGlobal Consortium, which is an association of graduate students 
and these programs that I mentioned, which also brings together 
people. They have a conference usually here in Washington in col- 
laboration between the AAAS and the National Academies to pro- 
vide an opportunity for younger people to explore this field, get into 
it if they’re interested, and some of them do. We at AAAS have 
hired on our staff several people who have been graduates of these 
programs — master’s degree graduates from these programs and 
some have been highly successful and are really leaders, young 
leaders in the field. 

So I am an advocate for this kind of education and I think we 
are doing it. I think it will be useful for Congress, and for Members 
of Congress if they were aware of this, to provide, I would say, 
moral support by speaking at their meetings and having staff at- 
tend and so on. So I am — I will leave it at that. 

Chairman LiPiNSKl. Well, thank you. Dr. Teich, and I had a — 
when we were going out for votes, I was getting in the elevator and 
someone who had been sitting in the audience, he went up and 
thanked me for having this, the hearing on SciSIP, and said, how 
do Members really become educated? How do you have the time? 
And I said, it is very, very difficult and what it really takes is a 
dedication to, you know, being educated, because the incentives, 
other than really wanting to do a good job and being interested in 
this topic, aren’t there. It is unfortunate. 

But the good thing is that we do have staff who are well edu- 
cated in these things and it leaves me to thank the staff for all 
their work that you do, and all the staff on the Science Committee 
do an excellent job so that we — help us to do a better job here, 
hopefully, on the Science and Technology Committee, help the 
Members do a better job. So I thank the staff for all the work that 
they do. 

With that I want to thank the witnesses for their testimony. The 
record will remain open for two weeks for additional statements 
from the Members and for answers to any follow up questions the 
Committee may ask of the witnesses. With that the witnesses are 
excused and the hearing is now adjourned. 

[Whereupon, at 4:23 p.m. the Subcommittee was adjourned.] 
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Answers to Post-Hearing Questions 

Responses by Dr. Julia Lane, Program Director of the Science of Science and Innova- 
tion Policy Program, National Science Foundation 

Questions submitted by Chairman Daniel Lipinski 

Ql. You describe in your testimony an effort by NSF to improve upon the way in 
which NSF interacts with its proposal and award portfolio. Can you please 
elaborate on this effort^ How might the new tools you are developing to be uti- 
lized in the development of future NSF budget proposals, new programs or other 
aspects of policy development at NSF? Also, can you please elaborate on the rel- 
evance of this effort to the broader impact criterion? 

Al. The SEE and CISE directorates have established a joint subcommittee of their 
directorate advisory committees that is exploring new ways to analyze and oversee 
NSF’s portfolio of proposals and awards. The subcommittee is developing a report 
that will be available to NSF leadership and the broader community in November 
2010. A particular focus of the report will be identifying tools to help NSF program 
staff better identify and support transformative and interdisciplinary research and 
gauge the broader impacts of NSF’s investments. The report will also advise NSF 
on how to better structure existing data, improve use of existing technologies to 
complement human expertise, and characterize its programmatic data. 

These new tools and resources have many potential uses in establishing, justi- 
fying, and implementing budgetary priorities for NSF. A principal aim is to assess 
the alignment of NSF’s priorities with emerging trends and opportunities in science 
and engineering research and education and to assess NSF’s impact on areas of na- 
tional priority. Other potential benefits include improving the efficiency of NSF’s 
core business processes by providing program officers with new resources for man- 
aging the merit review process. 
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Introduction 

The California Healthcare Institute (CHI) is the public policy association which unites over 275 
of our state's leading biotechnology, pharmaceutical, medical device, and diagnostics 
companies, premier universities and private research institutions, and venture capital firms to 
advocate policies that advance biomedical research, investment and innovation. We are 
grateful for the opportunity to provide comment on science and innovation policy. 

Additionally, we applaud the Research and Science Education Subcommittee for its continued 
attention to issues surrounding research and development programs, investment in science 
innovation, and workforce development. 

Background 

California's biomedical industry, employing 275,000 across the state, has been a powerful 
engine of economic growth for the Golden State for nearly 30 years. California has built a 
workforce and industry that is vital to the state's economy — and to the country's continued 
leadership in scientific, engineering and medical Innovation. Moreover, the partnerships and 
technology transfer agreements among California's biomedical companies and academic 
centers continue to nurture excellence in education here and innovations that are improving 
health, healthcare and quality of life the world over. 

Many of the jobs within the biomedical industry require high levels of specialized training, and 
the industry employs a substantial number of scientists, researchers and clinicians with 
doctorate degrees, both Ph.D.s and M.D.s. At the same time, opportunities exist within 
academia and companies for individuals with high school diplomas and undergraduate 
degrees. The industry also employs professionals with business, information technology, 
human resources and other areas of expertise. 

To continue to drive innovation in medicinal compounds and medical devices, researchers 
must be well-versed in math and science and insatiably curious about human conditions and 
diseases. Recruiting, managing and retaining today's brightest is a challenge that companies 
and universities spend significant resources addressing. 
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Yet the biomedical Industry must and does take the long view on workforce development. 
Through a number of programs, California companies are working to develop elementary and 
high school students' interest in math, science and engineering. Some programs are directed 
at the students themselves, while others are designed to give their teachers the tools and 
insights they need to make the sciences more compelling and relevant to young people. 

California's commercial and academic organizations also have collaborated in a number of 
ways to draw college students and working adults to life sciences careers. From biotechnology 
curricula at community colleges to business-focused skills training in post graduate programs, 
universities and industry are working together to ensure that prospective employees are 
equipped to succeed. 

An example is the Professional Science Master's (PSM) program at the California State 
University system. These programs concentrate in areas such as biotechnology, 
bioinformatics and biostatistics. The impetus to create the programs was in response to 
industry's resounding cry that academia does not provide enough graduates who can hit the 
ground running and bring immediate value. An online Master's of Science program in 
regulatory affairs is taught by regulatory professionals and teaches students to take a product 
from concept to FDA approval, including foreign regulatory approval. Industry professionals 
also teach our quality control program, which focuses on the processes involved in developing 
a company's standard operating procedures, such as bringing a product from early-stage 
clinical trials to product approval, and all of the extensive training and statistical analysis that 
must take place for successful results. 

Furthermore, researchers at universities and institutes (and within many companies 
throughout California) also work together to promote education and innovation through many 
avenues. Through support from the National Science Foundation (NSF), the National 
Institutes of Health (NIH) and other federal agencies, researchers are able to move 
discoveries from their labs toward the marketplace. These discoveries form the basis for 
technology transfer agreements that fuel the commercialization of medical breakthroughs, 
expand the world's scientific knowledge, improve public health and spearheads tomorrow's 
disease treatments. 

Conclusion 

In summary, CHI supports the Subcommittee's continued work relating to science and 
technology policy, particularly as it pertains to biomedical innovation. Additionally, we 
encourage the Subcommittee to continue to explore ways in which successful public-private 
partnerships have been formed and have fostered great success and enthusiasm in these 
fields. Again, we thank you for the opportunity to have our remarks be a part of the record. 
We look forward to working with you on this endeavor. 
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